-- Logs begin at Tue 2022-07-12 12:59:00 UTC, end at Tue 2022-07-12 13:59:05 UTC. -- Jul 12 13:27:31.673339 dv-cinder-797970-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 12 13:27:31 2022] *** Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: nodename: dv-cinder-797970-4-iscsi Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: machine: x86_64 Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: clock source: unix Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: pcre jit disabled Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: detected number of CPU cores: 4 Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: current working directory: / Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: detected binary path: /usr/bin/uwsgi-core Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: your processes number limit is 47680 Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: your memory page size is 4096 bytes Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: detected max file descriptor number: 2048 Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: lock engine: pthread robust mutexes Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: thunder lock: enabled Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 12 13:27:31.686123 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 12 13:27:31.729632 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: Python main interpreter initialized at 0x556fb3f7b9c0 Jul 12 13:27:31.729632 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: python threads support enabled Jul 12 13:27:31.729632 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: your server socket listen backlog is limited to 100 connections Jul 12 13:27:31.729632 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: your mercy for graceful operations on workers is 90 seconds Jul 12 13:27:31.730069 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: mapped 403077 bytes (393 KB) for 2 cores Jul 12 13:27:31.730069 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: *** Operational MODE: preforking *** Jul 12 13:27:31.730362 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: *** uWSGI is running in multiple interpreter mode *** Jul 12 13:27:31.730362 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: spawned uWSGI master process (pid: 80489) Jul 12 13:27:31.730574 dv-cinder-797970-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 12 13:27:31.731422 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: spawned uWSGI worker 1 (pid: 80499, cores: 1) Jul 12 13:27:31.731525 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: spawned uWSGI worker 2 (pid: 80500, cores: 1) Jul 12 13:27:31.735879 dv-cinder-797970-4-iscsi devstack@keystone.service[80489]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 12 13:27:37.200319 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 13:27:37.200319 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: from cryptography.utils import int_from_bytes Jul 12 13:27:37.201112 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 13:27:37.201112 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: from cryptography.utils import int_from_bytes Jul 12 13:27:37.201227 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 13:27:37.201227 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: from cryptography.utils import int_from_bytes Jul 12 13:27:37.201646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 13:27:37.201646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: from cryptography.utils import int_from_bytes Jul 12 13:27:37.249633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 12 13:27:37.249823 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 12 13:27:37.250087 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] command line args: [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 12 13:27:37.250372 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 12 13:27:37.250461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 12 13:27:37.250572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ================================================================================ {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 12 13:27:37.250654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 12 13:27:37.250811 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] command line args: [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 12 13:27:37.250942 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] admin_token = **** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.251093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 12 13:27:37.251174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] config_dir = [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.251258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ================================================================================ {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 12 13:27:37.251337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.251505 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] config_source = [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.251620 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] admin_token = **** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.251699 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] debug = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.251789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] config_dir = [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.251906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.252164 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.252164 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] config_source = [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.252285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] debug = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.252387 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.252449 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.252536 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.253237 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.253330 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.253403 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.253612 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.253704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.253881 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.254016 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] list_limit = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.254156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] log_config_append = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.254248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] list_limit = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.254312 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] log_config_append = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.254374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.254519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.254597 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] log_dir = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.254662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] log_dir = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.254731 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] log_file = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.254884 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] log_file = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.255294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.255384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.255565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.255653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.255817 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.255906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.256066 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.256153 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.256288 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.256370 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.256440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.256511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.256681 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.256767 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.256898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.257053 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.257183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.257331 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.257421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.257572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.257658 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.257808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.257892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.258087 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.258220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.258338 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.258502 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.258631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.258837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.259117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.259237 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.259347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] publish_errors = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.259527 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.259613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.259764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.259854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] publish_errors = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.259932 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.260008 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.260185 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.260272 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.260389 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.260483 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.260636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.260721 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] standard_threads = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.260871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.260947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.261104 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] standard_threads = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.261182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.261266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.261337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.261450 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] use_journal = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.261567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.261653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] use_json = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.261726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.261796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] use_stderr = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.261904 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] use_journal = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.261996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] use_syslog = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.262119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] use_json = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.262204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.262277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] use_stderr = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.262427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] use_syslog = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.262510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.262659 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:27:37.262747 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.262920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.262998 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.263159 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.263236 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.263411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.263498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.263655 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.263773 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.263893 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.264012 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.264131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.264257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.264372 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.264484 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.264602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.264756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.264898 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.265025 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.265139 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.265260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.265373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.265485 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.265597 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.265717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.265831 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.265943 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.266090 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.266204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.266329 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.266522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.266646 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.266766 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.266887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.267002 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.267115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.267237 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.267359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.267473 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.267585 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.267698 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.267810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.267931 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.268045 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.268158 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.268277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.269036 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.269167 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.269291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.269415 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.269530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.269647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.269761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.269880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.270042 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.270160 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.270160 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.270371 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.270485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.270606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.270721 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.270834 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.270946 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.271058 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.271177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.271300 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.271420 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.271536 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.271655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.271769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.271884 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.272005 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.272125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.272245 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.272358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.272477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.272592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.272781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.272897 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.273017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.273131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.273244 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.273363 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.273484 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.273606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.273721 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.273721 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.273932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.274071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.274195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.274310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.274423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.274536 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.274656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.274776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.274898 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.275012 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.275114 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.275209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.275308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.275402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.275516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.275611 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.275704 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.275704 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.275890 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.276021 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.276130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.276232 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.276334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.276451 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.276565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.276678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.276802 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.276917 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.277031 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.277144 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.277258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.277372 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.277495 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.277618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.277742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.277867 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.278032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.278163 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.278289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.278404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.278516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.278637 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.278758 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.278874 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.278994 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.279108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.279233 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.279349 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.279463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.279576 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.279690 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.279804 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.279926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.280042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.280155 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.280269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.280382 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.280508 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.280624 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.280737 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.280846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.280967 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.281088 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.281211 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.281327 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.281441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.281555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.281668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.281782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.281908 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.282043 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.282157 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.282271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.282385 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.282499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.282624 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.282741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.282854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.282974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.283090 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.283203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.283325 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.283441 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.283554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.283674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.283795 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.283911 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.284100 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.284218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.284332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.284458 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.284573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.284688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.284810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.284937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.285065 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.285188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.285303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.285417 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.285531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.285645 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.285767 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.285883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.286022 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.286139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.286264 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.286386 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.286509 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.286625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.286738 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.286738 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.286960 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.287076 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.287198 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.287315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.287429 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.287542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.287657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.287777 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.287901 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.288015 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] auth.external = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.288129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] auth.external = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.288242 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.288356 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.288479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.288624 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.288755 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.288871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.288985 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] auth.password = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.289099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] auth.password = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.289214 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] auth.token = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.289329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] auth.token = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.289452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.289567 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.289682 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.289797 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.289911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.290037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.290165 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.290295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.290414 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.290530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.290646 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.290761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.290884 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.291001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.291116 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] credential.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.291237 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] credential.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.291343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.291458 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.291583 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.291708 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.291830 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.291945 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.292060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.292175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.292298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.292422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.292546 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.292666 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.292783 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.292897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.293020 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.293143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.293259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.293380 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.293497 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.293612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.293738 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.293855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.293985 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.294102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.294217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.294332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.294460 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.294578 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.294693 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.294809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.294933 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] federation.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.295049 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] federation.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.295176 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.295293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.295408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.295523 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.295639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.295754 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.295881 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.295997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.296113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.296228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.296344 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.296459 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.296583 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.296701 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.296824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.296939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.297060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.297175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.297299 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.297414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.297529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.297644 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.297760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.297874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.298123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.298243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.298359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.298475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.298591 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.298713 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.298831 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.298946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.299061 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.299061 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.299276 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.299398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.299514 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.299514 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.299730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.299845 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.299960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.299960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.300186 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.300301 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.300415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.300529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.300643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.300766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.300884 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.300998 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.301113 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.301227 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.301330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.301453 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.301570 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.301684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.301803 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.301920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.302048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.302176 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.302294 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.302409 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.302523 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.302638 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.302807 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.302935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.303052 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.303166 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.303281 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.303394 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.303509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.303632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.303748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.303863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.303978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.304093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.304208 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.304331 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.304446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.304561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.304676 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.304790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.304904 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.305027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.305143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.305257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.305372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.305486 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.305606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.305730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.305846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.305979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.306098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.306213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.306328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.306455 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.306572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.306701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.306819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.306934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.307049 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.307173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.307304 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.307478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.307596 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.307711 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.307826 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.307950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.308067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.308182 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.308297 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.308420 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.308535 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.308667 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.308784 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.308900 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.309014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.309135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.309135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.309361 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.309478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.309593 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.309707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.309822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.309936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.310135 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.310252 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.310367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.310481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.310594 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.310717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.310833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.310948 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.311063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.311177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.311291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.311416 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.311532 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.311647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.311762 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.311882 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.311999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.312123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.312239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.312354 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.312470 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.312584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.312699 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.312822 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.312938 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.313053 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.313167 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.313281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.313396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.313519 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.313636 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.313750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.313865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.313998 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.314115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.314243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.314359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.314474 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.314589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.314704 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.314819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.314942 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.315057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.315173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.315783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.315900 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.316015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.316144 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.316261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.316376 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.316491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.316608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.316723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.316847 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.316962 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.317077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.317191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.317306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.317420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.317544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.317668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.317785 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.317900 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.318140 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.318257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.318385 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.318503 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.318618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.318734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.318849 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.318963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.319086 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] resource.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.319202 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.319317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.319431 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.319547 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.319662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] resource.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.319787 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.319903 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.320018 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.320133 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.320248 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.320362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.320485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.320601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.320717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.320717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.320933 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.321048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.321236 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.321354 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] role.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.321469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.321584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] role.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.321699 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.321817 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.321941 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.322093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.322288 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] role.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.322409 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.322525 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] role.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.322639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.322769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.322887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.323002 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.323117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.323232 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.323347 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.323470 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.323587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.323703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.323818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.323934 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.324058 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.324174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.324290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.324405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.324520 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.324635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.324758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.324875 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.324991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.325106 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.325106 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.325322 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.325445 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.325563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.325677 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.325792 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.325792 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.326044 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.326176 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.326293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.326408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.326523 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.326639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.326753 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.326877 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.326993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.327109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.327224 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.327339 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.327454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.327579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.327695 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.327810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.327926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.328042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.328158 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.328281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.328398 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.328513 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.328628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.328743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.328866 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.328989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.329105 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.329221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.329336 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.329451 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.329574 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.329697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] token.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.329813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.329928 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.330109 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.330226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.330353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.330471 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.330586 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] token.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.330700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.330816 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.331591 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.331721 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.331839 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.331958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.332072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.332187 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.332302 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.332431 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.332576 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.332698 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.332813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.332929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.333044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.333044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.333272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.333389 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.333504 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.333634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.333798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.333987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.334147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.334304 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.334463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.334463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.334776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.334956 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.335116 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.connection = **** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.335285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.335452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.335611 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.335766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.335766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.336058 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.336225 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.336382 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.connection = **** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.336552 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.336720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.336876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.337033 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.337189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.337354 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.337546 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.337758 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.338046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.338231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.338421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.338631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.338824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.339003 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.339127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.339244 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.339361 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.339487 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.339604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.339722 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.339837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.339952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.340068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.340191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.340308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.340423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.340537 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.340652 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.340768 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 12 13:27:37.340893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.340893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.340893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.340893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:27:37.340893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 12 13:27:37.369464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.369693 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.369781 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.369859 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.369934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.370093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.378874 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.379104 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.407497 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.407720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.407878 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.408114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.552327 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.552689 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.602649 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.602914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.603312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.603552 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.669300 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.669487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.669574 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.669653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.669752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.669832 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.669941 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.670047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.670183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.670282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.670402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.670485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.696664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.696893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.697087 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.697213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.697328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.697417 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.697556 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.697653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.697797 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.697905 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.698040 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.698127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.698312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.698477 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.709609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.709984 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.722899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.723133 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.723247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.723360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.723442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.723582 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.723665 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.723802 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.723887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.724018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.724100 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.724250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.724330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.724463 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:27:37.742604 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.742753 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.742895 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 13:27:37.742981 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 13:27:37.743136 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.743339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 13:27:37.743420 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.743556 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 13:27:37.743638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.743824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.743916 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.744082 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.744197 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.744339 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.744430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.744594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.756586 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.756703 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.756881 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.756970 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.757222 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.757338 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.757495 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.757582 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.766004 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.766173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.769348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.769447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.769637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.769723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.769918 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.770033 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.770222 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.770305 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.770502 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.770600 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.770756 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.770839 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.771025 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.771118 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.771288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.771377 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.771551 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.771634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.771818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.771898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.772080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.772169 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.798286 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.798500 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.800403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.800653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.800754 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.800936 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.806069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.806289 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.806388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.806570 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.806664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.807049 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.813941 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.814245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.814383 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.814677 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.819303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.819585 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.819751 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.819867 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.820045 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.820194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.820315 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.820394 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.820602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.820684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.820879 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.820963 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.821151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.821231 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.821419 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.821688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.839892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.840177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.840666 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.840951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.845031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.845572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.848723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.849015 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.849274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.849572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.854157 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.854753 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.857741 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.858412 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.861398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.861688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.861992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.862635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.862841 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.862955 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.863120 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.863208 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.863390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.863476 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.863763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.864042 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.883124 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.883406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.883670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.883938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.884024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.884276 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.884543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.884812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.891811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.892755 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.893528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.893800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.894528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.894805 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.897052 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.897328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.897607 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.898004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.898093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.898295 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.898381 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.898571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.898848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.899177 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.911978 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.912250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.912527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.912800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.913056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.913151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.913338 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.913423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.913605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.913688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.913881 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.914190 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.915912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.916203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.933320 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.934632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.936585 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.937899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.939883 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.941282 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.941517 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.943054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.943301 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.943576 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.943860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.944702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.944983 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.945272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.951173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.952603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.952755 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.954257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.956028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.956313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.956590 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.956870 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.957519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.957808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.958227 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.958515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.966034 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.966315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.966590 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.967647 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.967936 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.968217 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.974947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.975223 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.975495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.975765 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.976044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.976319 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.976585 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.976678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:27:37.976855 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.976939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.977158 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.977238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.977436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.977519 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.977709 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.977799 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.978003 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.978112 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.978314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.978407 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.978544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.978663 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.978937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.979210 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.979498 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.979764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.980040 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:37.980307 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:27:38.017192 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x556fb3f7b9c0 pid: 80499 (default app) Jul 12 13:27:38.018749 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x556fb3f7b9c0 pid: 80500 (default app) Jul 12 13:27:38.020669 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a35ff-b493-4522-b85b-8ba6902ea418 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:38.020960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a35ff-b493-4522-b85b-8ba6902ea418 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:38.021239 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a35ff-b493-4522-b85b-8ba6902ea418 None None] PATH_INFO: `/v3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:38.025432 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Tue Jul 12 13:27:38 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:45.644848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c5243-4e8d-423c-b3e6-7d2972e5ce0c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:45.645448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c5243-4e8d-423c-b3e6-7d2972e5ce0c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:45.645448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c5243-4e8d-423c-b3e6-7d2972e5ce0c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:45.649830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:27:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:27:45.655475 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0aa2c-5c09-4d9c-8616-9ea4d099e422 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:45.655922 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0aa2c-5c09-4d9c-8616-9ea4d099e422 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:45.656011 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0aa2c-5c09-4d9c-8616-9ea4d099e422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:45.725242 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: INFO dbcounter [None req-63f0aa2c-5c09-4d9c-8616-9ea4d099e422 None None] Registered counter for database keystone Jul 12 13:27:45.746463 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG oslo_db.sqlalchemy.engines [None req-63f0aa2c-5c09-4d9c-8616-9ea4d099e422 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80499) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 12 13:27:45.753358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writer thread running {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 12 13:27:45.940339 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG passlib.handlers.bcrypt [None req-63f0aa2c-5c09-4d9c-8616-9ea4d099e422 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80499) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 12 13:27:45.940668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG passlib.handlers.bcrypt [None req-63f0aa2c-5c09-4d9c-8616-9ea4d099e422 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80499) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 12 13:27:45.995859 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-63f0aa2c-5c09-4d9c-8616-9ea4d099e422 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:45.997501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-63f0aa2c-5c09-4d9c-8616-9ea4d099e422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:46.130725 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:27:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 12 13:27:46.320756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c5243-4e8d-423c-b3e6-7d2972e5ce0c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:46.390841 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: INFO dbcounter [None req-dd1c5243-4e8d-423c-b3e6-7d2972e5ce0c None None] Registered counter for database keystone Jul 12 13:27:46.410878 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG oslo_db.sqlalchemy.engines [None req-dd1c5243-4e8d-423c-b3e6-7d2972e5ce0c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80500) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 12 13:27:46.417323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writer thread running {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 12 13:27:46.536156 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c5243-4e8d-423c-b3e6-7d2972e5ce0c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:46.540490 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ed1a1-52d9-4713-82e1-b94120559c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:46.541150 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ed1a1-52d9-4713-82e1-b94120559c40 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:46.541434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ed1a1-52d9-4713-82e1-b94120559c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:46.541709 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ed1a1-52d9-4713-82e1-b94120559c40 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:46.550062 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ed1a1-52d9-4713-82e1-b94120559c40 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:46.602363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e33ed1a1-52d9-4713-82e1-b94120559c40 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 12 13:27:46.602702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ed1a1-52d9-4713-82e1-b94120559c40 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:46.612788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Tue Jul 12 13:27:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:27:46.618797 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0aa2c-5c09-4d9c-8616-9ea4d099e422 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:46.633566 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0aa2c-5c09-4d9c-8616-9ea4d099e422 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:46.638112 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5c902-744d-4417-9bb5-c92fb9a88eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:46.639247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5c902-744d-4417-9bb5-c92fb9a88eed admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:46.639531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5c902-744d-4417-9bb5-c92fb9a88eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:46.639805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5c902-744d-4417-9bb5-c92fb9a88eed admin admin] PATH_INFO: `/v3/projects/8e45bcca33094251b8a3ed60293c2eb5` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:46.641313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5c902-744d-4417-9bb5-c92fb9a88eed admin admin] RBAC: Authorizing `identity:get_project(project_id=8e45bcca33094251b8a3ed60293c2eb5)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:46.708398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: WARNING keystone.common.rbac_enforcer.enforcer [None req-87f5c902-744d-4417-9bb5-c92fb9a88eed admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 12 13:27:46.709059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5c902-744d-4417-9bb5-c92fb9a88eed admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:46.713504 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Tue Jul 12 13:27:46 2022] GET /identity/v3/projects/8e45bcca33094251b8a3ed60293c2eb5 => generated 344 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:27:48.808980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f576ef-80f8-42f2-a289-0fd337ae0bfe None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:48.809565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f576ef-80f8-42f2-a289-0fd337ae0bfe None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:48.809565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f576ef-80f8-42f2-a289-0fd337ae0bfe None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:48.813679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:27:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:27:48.819985 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a7465-169c-4a0e-9976-a7b198c4c4d9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:48.820565 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a7465-169c-4a0e-9976-a7b198c4c4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:48.820565 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a7465-169c-4a0e-9976-a7b198c4c4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:48.840174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4e3a7465-169c-4a0e-9976-a7b198c4c4d9 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:48.841747 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4e3a7465-169c-4a0e-9976-a7b198c4c4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:48.850091 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:27:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 12 13:27:48.872762 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f576ef-80f8-42f2-a289-0fd337ae0bfe None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:48.885899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f576ef-80f8-42f2-a289-0fd337ae0bfe None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:48.890218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0d304-9ea0-4ec3-875d-dd1d136182de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:48.890857 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0d304-9ea0-4ec3-875d-dd1d136182de admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:48.891145 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0d304-9ea0-4ec3-875d-dd1d136182de admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:48.891424 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0d304-9ea0-4ec3-875d-dd1d136182de admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:48.899407 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0d304-9ea0-4ec3-875d-dd1d136182de admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:48.901218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0d304-9ea0-4ec3-875d-dd1d136182de admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:48.910536 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Tue Jul 12 13:27:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:27:48.916295 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a7465-169c-4a0e-9976-a7b198c4c4d9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:48.926298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a7465-169c-4a0e-9976-a7b198c4c4d9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:48.930790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dbb53-8468-477c-a547-fb2ae6b8ef3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:48.931673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dbb53-8468-477c-a547-fb2ae6b8ef3d admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:48.931960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dbb53-8468-477c-a547-fb2ae6b8ef3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:48.932240 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dbb53-8468-477c-a547-fb2ae6b8ef3d admin admin] PATH_INFO: `/v3/users/0ed92bd32dc24cfabfcf0a70a4518dcf` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:48.933655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dbb53-8468-477c-a547-fb2ae6b8ef3d admin admin] RBAC: Authorizing `identity:get_user(user_id=0ed92bd32dc24cfabfcf0a70a4518dcf)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:48.935548 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dbb53-8468-477c-a547-fb2ae6b8ef3d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:48.937183 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Tue Jul 12 13:27:48 2022] GET /identity/v3/users/0ed92bd32dc24cfabfcf0a70a4518dcf => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:51.059380 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5538c-fa08-490a-80b3-ab8ded5ffde1 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:51.059920 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5538c-fa08-490a-80b3-ab8ded5ffde1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:51.059920 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5538c-fa08-490a-80b3-ab8ded5ffde1 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:51.061290 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:27:51.067445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fefb36-caee-4217-bf11-a9781977ff9a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:51.067914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fefb36-caee-4217-bf11-a9781977ff9a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:51.068177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fefb36-caee-4217-bf11-a9781977ff9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:51.086440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d3fefb36-caee-4217-bf11-a9781977ff9a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:51.087662 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d3fefb36-caee-4217-bf11-a9781977ff9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:51.127570 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479e32d-676f-4f3b-b693-020dfe0e21eb None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:51.127863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479e32d-676f-4f3b-b693-020dfe0e21eb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:51.128152 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479e32d-676f-4f3b-b693-020dfe0e21eb None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:51.129843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:27:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:27:51.135834 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a418f-3849-40dc-a0de-3978343b289e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:51.136124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a418f-3849-40dc-a0de-3978343b289e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:51.136407 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a418f-3849-40dc-a0de-3978343b289e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:51.180079 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:27:51.180799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG passlib.handlers.bcrypt [None req-4b7a418f-3849-40dc-a0de-3978343b289e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80500) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 12 13:27:51.181294 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG passlib.handlers.bcrypt [None req-4b7a418f-3849-40dc-a0de-3978343b289e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80500) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 12 13:27:51.202966 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fefb36-caee-4217-bf11-a9781977ff9a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:51.228537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4b7a418f-3849-40dc-a0de-3978343b289e None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:51.230082 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:51.230238 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4b7a418f-3849-40dc-a0de-3978343b289e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:51.316592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fefb36-caee-4217-bf11-a9781977ff9a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:51.339083 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:27:51.355786 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f799667-16ac-4560-8845-ae7e3640cebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:51.356620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f799667-16ac-4560-8845-ae7e3640cebd None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:51.356903 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f799667-16ac-4560-8845-ae7e3640cebd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:51.357183 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f799667-16ac-4560-8845-ae7e3640cebd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:51.364886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a418f-3849-40dc-a0de-3978343b289e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:51.374560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f799667-16ac-4560-8845-ae7e3640cebd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:27:51.388386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-5f799667-16ac-4560-8845-ae7e3640cebd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:27:51.388386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:27:51.388386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:27:51.388386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:27:51.388386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:27:51.388386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:27:51.388386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:27:51.388386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:27:51.388386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:27:51.388386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:27:51.388386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:27:51.388386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:27:51.388386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:27:51.388386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:27:51.388386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:27:51.388386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:27:51.389498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:27:51.389498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:27:51.389498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:27:51.389498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:27:51.389498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:27:51.389498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:27:51.389498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:27:51.389498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:27:51.389498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:27:51.389498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:27:51.389498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:27:51.389498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:27:51.389498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:27:51.389498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:27:51.389498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:27:51.389498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:27:51.390572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:27:51.390572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:27:51.390572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:27:51.390572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:27:51.390572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:27:51.390572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:27:51.390572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:27:51.390572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:27:51.390572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:27:51.390572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:27:51.390572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:27:51.390572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:27:51.390572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:27:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:27:51.396234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f799667-16ac-4560-8845-ae7e3640cebd None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:51.484198 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a418f-3849-40dc-a0de-3978343b289e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:51.487378 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:51.487618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:51.513418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f799667-16ac-4560-8845-ae7e3640cebd None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:51.530084 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03980fc9-5abf-46da-bc7d-692b00dadeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:51.530999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03980fc9-5abf-46da-bc7d-692b00dadeac None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:51.531299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03980fc9-5abf-46da-bc7d-692b00dadeac None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:51.531573 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03980fc9-5abf-46da-bc7d-692b00dadeac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:51.557693 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098ff9f-7ecd-4469-910f-c7a067ae6b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:51.558409 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098ff9f-7ecd-4469-910f-c7a067ae6b98 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:51.558695 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098ff9f-7ecd-4469-910f-c7a067ae6b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:51.558973 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098ff9f-7ecd-4469-910f-c7a067ae6b98 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:51.559733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098ff9f-7ecd-4469-910f-c7a067ae6b98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:51.560039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098ff9f-7ecd-4469-910f-c7a067ae6b98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:51.561469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098ff9f-7ecd-4469-910f-c7a067ae6b98 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:51.574769 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:27:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:27:51.580654 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098ff9f-7ecd-4469-910f-c7a067ae6b98 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:51.607970 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03980fc9-5abf-46da-bc7d-692b00dadeac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:51.609544 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03980fc9-5abf-46da-bc7d-692b00dadeac None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:51.691054 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098ff9f-7ecd-4469-910f-c7a067ae6b98 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:51.728211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:27:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 13:27:51.732100 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c0cfd-c66b-4dc5-880a-38efd25a1472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:51.733334 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c0cfd-c66b-4dc5-880a-38efd25a1472 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:51.733629 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c0cfd-c66b-4dc5-880a-38efd25a1472 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:51.733906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c0cfd-c66b-4dc5-880a-38efd25a1472 None admin] PATH_INFO: `/v3/users/0ed92bd32dc24cfabfcf0a70a4518dcf` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:51.734244 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03980fc9-5abf-46da-bc7d-692b00dadeac None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:51.735336 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c0cfd-c66b-4dc5-880a-38efd25a1472 None admin] RBAC: Authorizing `identity:get_user(user_id=0ed92bd32dc24cfabfcf0a70a4518dcf)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:51.736948 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c0cfd-c66b-4dc5-880a-38efd25a1472 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:51.737747 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:51.738922 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:27:51 2022] GET /identity/v3/users/0ed92bd32dc24cfabfcf0a70a4518dcf => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:27:51.744662 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c0cfd-c66b-4dc5-880a-38efd25a1472 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:51.761778 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:51.861997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03980fc9-5abf-46da-bc7d-692b00dadeac None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:51.868565 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c0cfd-c66b-4dc5-880a-38efd25a1472 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:51.899452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69ade3-f6c6-4ea9-a4d9-f4f5aee4c47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:51.900277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69ade3-f6c6-4ea9-a4d9-f4f5aee4c47e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:51.900563 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69ade3-f6c6-4ea9-a4d9-f4f5aee4c47e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:51.900839 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69ade3-f6c6-4ea9-a4d9-f4f5aee4c47e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:51.905508 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545f6fe-cf53-4cfe-9824-1a54c24afd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:51.906447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545f6fe-cf53-4cfe-9824-1a54c24afd2d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:51.906798 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545f6fe-cf53-4cfe-9824-1a54c24afd2d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:51.907081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545f6fe-cf53-4cfe-9824-1a54c24afd2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:51.908392 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545f6fe-cf53-4cfe-9824-1a54c24afd2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:51.909879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545f6fe-cf53-4cfe-9824-1a54c24afd2d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:51.911613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 10/18] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:27:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:27:51.917605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545f6fe-cf53-4cfe-9824-1a54c24afd2d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:51.922978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69ade3-f6c6-4ea9-a4d9-f4f5aee4c47e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:51.925046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69ade3-f6c6-4ea9-a4d9-f4f5aee4c47e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:51.947363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-cf69ade3-f6c6-4ea9-a4d9-f4f5aee4c47e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:27:51.947363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:27:51.947363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:27:51.947363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:27:51.947363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:27:51.947363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:27:51.947363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:27:51.947363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:27:51.947363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:27:51.947363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:27:51.947363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:27:51.947363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:27:51.947363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:27:51.947363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:27:51.947363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:27:51.948239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:27:51.948239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:27:51.948239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:27:51.948239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:27:51.948239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:27:51.948239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:27:51.948239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:27:51.948239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:27:51.948239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:27:51.948239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:27:51.948239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:27:51.948239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:27:51.948239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:27:51.948239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:27:51.948239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:27:51.948239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:27:51.948239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:27:51.949032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:27:51.949032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:27:51.949032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:27:51.949032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:27:51.949032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 9/19] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:27:51.954358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69ade3-f6c6-4ea9-a4d9-f4f5aee4c47e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:52.000625 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:52.011307 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:52.020810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545f6fe-cf53-4cfe-9824-1a54c24afd2d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:52.054396 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497cbd3-f05c-441c-84f5-c5b710369492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:52.054770 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69ade3-f6c6-4ea9-a4d9-f4f5aee4c47e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:52.055175 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497cbd3-f05c-441c-84f5-c5b710369492 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:52.055629 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497cbd3-f05c-441c-84f5-c5b710369492 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:52.056082 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497cbd3-f05c-441c-84f5-c5b710369492 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:52.057265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497cbd3-f05c-441c-84f5-c5b710369492 None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.domain.id=default, user.id=0ed92bd32dc24cfabfcf0a70a4518dcf) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:52.057580 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497cbd3-f05c-441c-84f5-c5b710369492 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:52.059130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497cbd3-f05c-441c-84f5-c5b710369492 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:52.066902 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Tue Jul 12 13:27:51 2022] GET /identity/v3/role_assignments?user.id=0ed92bd32dc24cfabfcf0a70a4518dcf&scope.domain.id=default&role.id=864a6966630544e8bd68bcafba905c6f => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:27:52.090865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a738e4-d85d-4b74-9157-b71f90884b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:52.091550 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a738e4-d85d-4b74-9157-b71f90884b44 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:52.091835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a738e4-d85d-4b74-9157-b71f90884b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:52.092109 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a738e4-d85d-4b74-9157-b71f90884b44 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:52.092865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a738e4-d85d-4b74-9157-b71f90884b44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:52.093176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a738e4-d85d-4b74-9157-b71f90884b44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:52.094656 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a738e4-d85d-4b74-9157-b71f90884b44 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:52.114513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:27:54.496914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b7991-8966-401f-97d6-db7c82d726a1 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:54.497949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b7991-8966-401f-97d6-db7c82d726a1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:54.498574 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b7991-8966-401f-97d6-db7c82d726a1 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:54.500178 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:27:54.508013 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830f2f0-bac9-45b8-ab00-8875cbee2061 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:54.508496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830f2f0-bac9-45b8-ab00-8875cbee2061 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:54.508684 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830f2f0-bac9-45b8-ab00-8875cbee2061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:54.534029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4830f2f0-bac9-45b8-ab00-8875cbee2061 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:54.535238 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4830f2f0-bac9-45b8-ab00-8875cbee2061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:54.555512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f007fe-1800-48b7-82be-304bf03a2ca2 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:54.556144 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f007fe-1800-48b7-82be-304bf03a2ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:54.556625 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f007fe-1800-48b7-82be-304bf03a2ca2 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:54.558272 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 13/23] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:27:54.563708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096974a5-c95a-4863-922d-2c95b9696b38 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:54.564768 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096974a5-c95a-4863-922d-2c95b9696b38 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:54.565250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096974a5-c95a-4863-922d-2c95b9696b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:54.596079 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-096974a5-c95a-4863-922d-2c95b9696b38 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:54.597697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-096974a5-c95a-4863-922d-2c95b9696b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:54.659465 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 11/24] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:27:54.705708 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830f2f0-bac9-45b8-ab00-8875cbee2061 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:54.750827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 14/25] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:27:54.788169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096974a5-c95a-4863-922d-2c95b9696b38 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:54.808177 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:54.918153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830f2f0-bac9-45b8-ab00-8875cbee2061 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:54.988826 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af145acd-60be-4ca1-9b6b-e029da496cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:54.989995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af145acd-60be-4ca1-9b6b-e029da496cce None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:54.990617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af145acd-60be-4ca1-9b6b-e029da496cce None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:54.991110 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af145acd-60be-4ca1-9b6b-e029da496cce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.007619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:55.017405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af145acd-60be-4ca1-9b6b-e029da496cce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:27:55.028979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-af145acd-60be-4ca1-9b6b-e029da496cce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:27:55.028979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:27:55.028979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:27:55.028979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:27:55.028979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:27:55.028979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:27:55.028979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:27:55.028979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:27:55.028979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:27:55.028979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:27:55.028979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:27:55.028979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:27:55.028979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:27:55.028979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:27:55.028979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:27:55.028979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:27:55.029847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:27:55.029847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:27:55.029847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:27:55.029847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:27:55.029847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:27:55.029847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:27:55.029847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:27:55.029847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:27:55.029847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:27:55.029847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:27:55.029847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:27:55.029847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:27:55.029847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:27:55.029847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:27:55.029847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:27:55.029847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:27:55.030630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:27:55.030630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:27:55.030630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:27:55.030630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:27:55.030630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:27:55.030630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:27:55.030630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:27:55.030630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:27:55.030630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:27:55.030630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:27:55.030630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:27:55.030630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:27:55.044640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 12/26] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:27:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:27:55.049232 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d7004-760a-4ecc-8c4b-b37c435ac6f2 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.050415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d7004-760a-4ecc-8c4b-b37c435ac6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.051048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d7004-760a-4ecc-8c4b-b37c435ac6f2 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.059162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 13/27] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:27:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:27:55.061383 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d7004-760a-4ecc-8c4b-b37c435ac6f2 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:55.070001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096974a5-c95a-4863-922d-2c95b9696b38 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:55.151264 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf6e8e-eab6-41be-b7d1-fec481b06532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:55.152157 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf6e8e-eab6-41be-b7d1-fec481b06532 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.152377 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf6e8e-eab6-41be-b7d1-fec481b06532 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.152665 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf6e8e-eab6-41be-b7d1-fec481b06532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.173829 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf6e8e-eab6-41be-b7d1-fec481b06532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:55.179421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf6e8e-eab6-41be-b7d1-fec481b06532 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:55.199444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d7004-760a-4ecc-8c4b-b37c435ac6f2 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:55.208648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:55.209790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-43cf6e8e-eab6-41be-b7d1-fec481b06532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:27:55.209790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:27:55.209790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:27:55.209790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:27:55.209790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:27:55.209790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:27:55.209790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:27:55.209790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:27:55.209790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:27:55.209790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:27:55.209790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:27:55.209790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:27:55.209790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:27:55.209790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:27:55.209790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:27:55.211618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:27:55.211618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:27:55.211618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:27:55.211618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:27:55.211618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:27:55.211618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:27:55.211618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:27:55.211618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:27:55.211618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:27:55.211618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:27:55.211618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:27:55.211618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:27:55.211618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:27:55.211618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:27:55.211618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:27:55.211618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:27:55.211618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:27:55.212423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:27:55.212423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:27:55.212423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:27:55.212423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:27:55.212423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 15/28] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:27:55.213584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5141770-7b5b-40e0-8607-3e7e1d2b45c5 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.214060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5141770-7b5b-40e0-8607-3e7e1d2b45c5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.214485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5141770-7b5b-40e0-8607-3e7e1d2b45c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.247049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d5141770-7b5b-40e0-8607-3e7e1d2b45c5 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:55.249646 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d5141770-7b5b-40e0-8607-3e7e1d2b45c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:55.257548 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c661b60-1309-4c96-9b19-26f5978b7119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:55.258555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c661b60-1309-4c96-9b19-26f5978b7119 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.258992 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c661b60-1309-4c96-9b19-26f5978b7119 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.259419 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c661b60-1309-4c96-9b19-26f5978b7119 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.260329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c661b60-1309-4c96-9b19-26f5978b7119 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:55.261267 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c661b60-1309-4c96-9b19-26f5978b7119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:55.262894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c661b60-1309-4c96-9b19-26f5978b7119 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:55.279703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 14/29] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:27:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:55.281546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c661b60-1309-4c96-9b19-26f5978b7119 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:55.332809 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 16/30] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:27:55.334633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a5c3f-6dce-4505-8721-dc28779b2394 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.335213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a5c3f-6dce-4505-8721-dc28779b2394 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.335712 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a5c3f-6dce-4505-8721-dc28779b2394 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.337366 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 17/31] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:27:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:27:55.342901 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a5c3f-6dce-4505-8721-dc28779b2394 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:55.411377 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c661b60-1309-4c96-9b19-26f5978b7119 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:55.418652 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:55.464795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38deea-1984-4856-bc76-5fedd5bc6dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:55.466209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38deea-1984-4856-bc76-5fedd5bc6dba None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.467067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38deea-1984-4856-bc76-5fedd5bc6dba None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.467913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38deea-1984-4856-bc76-5fedd5bc6dba None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.469459 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38deea-1984-4856-bc76-5fedd5bc6dba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:55.470263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38deea-1984-4856-bc76-5fedd5bc6dba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:55.472000 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38deea-1984-4856-bc76-5fedd5bc6dba None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:55.474970 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a5c3f-6dce-4505-8721-dc28779b2394 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:55.488383 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 15/32] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:55.490467 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b6d27-e0c1-4d27-b957-5be4104bf333 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.490980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b6d27-e0c1-4d27-b957-5be4104bf333 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.491454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b6d27-e0c1-4d27-b957-5be4104bf333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.511575 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f40b6d27-e0c1-4d27-b957-5be4104bf333 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:55.513559 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f40b6d27-e0c1-4d27-b957-5be4104bf333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:55.525302 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cc768-6480-4c61-b61d-a3f5e7ac8247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:55.526280 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cc768-6480-4c61-b61d-a3f5e7ac8247 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.526569 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cc768-6480-4c61-b61d-a3f5e7ac8247 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.526839 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cc768-6480-4c61-b61d-a3f5e7ac8247 None admin] PATH_INFO: `/v3/users/0ed92bd32dc24cfabfcf0a70a4518dcf` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.528583 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cc768-6480-4c61-b61d-a3f5e7ac8247 None admin] RBAC: Authorizing `identity:get_user(user_id=0ed92bd32dc24cfabfcf0a70a4518dcf)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:55.530641 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cc768-6480-4c61-b61d-a3f5e7ac8247 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:55.532366 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 18/33] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:27:55 2022] GET /identity/v3/users/0ed92bd32dc24cfabfcf0a70a4518dcf => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:55.533167 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cc768-6480-4c61-b61d-a3f5e7ac8247 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:55.547269 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:55.606098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 16/34] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:27:55.607012 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b6d27-e0c1-4d27-b957-5be4104bf333 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:55.654769 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cc768-6480-4c61-b61d-a3f5e7ac8247 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:55.686374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:55.700775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eeb3b0-5f05-4f0e-a67b-ea07a0001fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:55.701646 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eeb3b0-5f05-4f0e-a67b-ea07a0001fbd None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.702059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eeb3b0-5f05-4f0e-a67b-ea07a0001fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.702465 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eeb3b0-5f05-4f0e-a67b-ea07a0001fbd None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.703369 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eeb3b0-5f05-4f0e-a67b-ea07a0001fbd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:55.704928 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eeb3b0-5f05-4f0e-a67b-ea07a0001fbd None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:55.741476 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b6d27-e0c1-4d27-b957-5be4104bf333 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:55.780464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc6eff-c1c6-419f-9077-6b4d69aff5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:55.781249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc6eff-c1c6-419f-9077-6b4d69aff5a3 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.781685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc6eff-c1c6-419f-9077-6b4d69aff5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.782098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc6eff-c1c6-419f-9077-6b4d69aff5a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.783357 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc6eff-c1c6-419f-9077-6b4d69aff5a3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:55.785234 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc6eff-c1c6-419f-9077-6b4d69aff5a3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:55.840295 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 19/35] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Jul 12 13:27:55 2022] POST /identity/v3/roles => generated 225 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:55.841615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eeb3b0-5f05-4f0e-a67b-ea07a0001fbd None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:55.962969 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 17/36] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jul 12 13:27:55 2022] POST /identity/v3/projects => generated 301 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:55.964031 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc6eff-c1c6-419f-9077-6b4d69aff5a3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:55.987150 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:56.005432 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eeb3b0-5f05-4f0e-a67b-ea07a0001fbd None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:56.050770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a8596-7ff6-40f5-a685-44b72f2045c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:56.051625 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a8596-7ff6-40f5-a685-44b72f2045c2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:56.051915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a8596-7ff6-40f5-a685-44b72f2045c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:56.052196 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a8596-7ff6-40f5-a685-44b72f2045c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:56.053564 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a8596-7ff6-40f5-a685-44b72f2045c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:56.055396 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a8596-7ff6-40f5-a685-44b72f2045c2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:56.057430 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 20/37] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:27:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:56.059016 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00393a55-8196-49f0-ad96-fa80713bd452 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:56.059314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00393a55-8196-49f0-ad96-fa80713bd452 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:56.059598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00393a55-8196-49f0-ad96-fa80713bd452 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:56.061135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 21/38] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:27:56.062390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41799cf2-be81-4ba3-88e3-f27c1663125e None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:56.062695 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41799cf2-be81-4ba3-88e3-f27c1663125e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:56.063040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41799cf2-be81-4ba3-88e3-f27c1663125e None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:56.064465 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 22/39] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:27:56.071313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41799cf2-be81-4ba3-88e3-f27c1663125e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:56.119631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc6eff-c1c6-419f-9077-6b4d69aff5a3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:56.144597 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:56.162352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a423fac-86e6-498d-8c41-d7a6b43fdc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:56.163303 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a423fac-86e6-498d-8c41-d7a6b43fdc76 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:56.163607 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a423fac-86e6-498d-8c41-d7a6b43fdc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:56.163890 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a423fac-86e6-498d-8c41-d7a6b43fdc76 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:56.164659 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a423fac-86e6-498d-8c41-d7a6b43fdc76 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:56.166298 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a423fac-86e6-498d-8c41-d7a6b43fdc76 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:56.186172 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 18/40] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Jul 12 13:27:55 2022] POST /identity/v3/roles => generated 219 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:56.188513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf10d7-b240-4c3d-b3ab-795c3234e89e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:56.188867 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf10d7-b240-4c3d-b3ab-795c3234e89e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:56.189145 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf10d7-b240-4c3d-b3ab-795c3234e89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:56.202687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41799cf2-be81-4ba3-88e3-f27c1663125e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:56.211404 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-dbdf10d7-b240-4c3d-b3ab-795c3234e89e None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:56.213229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-dbdf10d7-b240-4c3d-b3ab-795c3234e89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:56.297749 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:56.310254 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 19/41] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:27:56.312079 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34929a86-2905-4969-89b0-2bf5315f3fd0 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:56.312517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34929a86-2905-4969-89b0-2bf5315f3fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:56.312948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34929a86-2905-4969-89b0-2bf5315f3fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:56.334042 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-34929a86-2905-4969-89b0-2bf5315f3fd0 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:56.335547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-34929a86-2905-4969-89b0-2bf5315f3fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:56.353557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b032d-6439-4b3e-8fb4-350affdb21fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:56.354988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b032d-6439-4b3e-8fb4-350affdb21fb None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:56.354988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b032d-6439-4b3e-8fb4-350affdb21fb None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:56.354988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b032d-6439-4b3e-8fb4-350affdb21fb None admin] PATH_INFO: `/v3/domains/default/users/0ed92bd32dc24cfabfcf0a70a4518dcf/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:56.357873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b032d-6439-4b3e-8fb4-350affdb21fb None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=0ed92bd32dc24cfabfcf0a70a4518dcf, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:56.359389 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b032d-6439-4b3e-8fb4-350affdb21fb None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:56.381417 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 23/42] 10.222.0.52 () {58 vars in 1361 bytes} [Tue Jul 12 13:27:56 2022] PUT /identity/v3/domains/default/users/0ed92bd32dc24cfabfcf0a70a4518dcf/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:56.382660 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b032d-6439-4b3e-8fb4-350affdb21fb None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:56.430800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 20/43] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:27:56.453850 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34929a86-2905-4969-89b0-2bf5315f3fd0 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:56.521650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b032d-6439-4b3e-8fb4-350affdb21fb None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:56.541775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:56.568609 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660f474-f3f6-4b40-9830-04273226e726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:56.569665 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660f474-f3f6-4b40-9830-04273226e726 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:56.569953 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660f474-f3f6-4b40-9830-04273226e726 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:56.570260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660f474-f3f6-4b40-9830-04273226e726 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:56.571449 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660f474-f3f6-4b40-9830-04273226e726 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:56.572934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660f474-f3f6-4b40-9830-04273226e726 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:56.574609 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 24/44] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:27:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:56.576296 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34929a86-2905-4969-89b0-2bf5315f3fd0 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:56.580572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660f474-f3f6-4b40-9830-04273226e726 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:56.608165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:56.614397 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255825cd-b7fd-43bf-9717-8037d6ecc1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:56.615074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255825cd-b7fd-43bf-9717-8037d6ecc1b1 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:56.615360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255825cd-b7fd-43bf-9717-8037d6ecc1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:56.615639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255825cd-b7fd-43bf-9717-8037d6ecc1b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:56.616407 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255825cd-b7fd-43bf-9717-8037d6ecc1b1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:56.617909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255825cd-b7fd-43bf-9717-8037d6ecc1b1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:56.634235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 21/45] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Jul 12 13:27:56 2022] POST /identity/v3/roles => generated 223 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:56.706492 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660f474-f3f6-4b40-9830-04273226e726 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:56.743574 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6579dd4-0417-4a6e-b89b-1c54a62d7d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:56.744254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6579dd4-0417-4a6e-b89b-1c54a62d7d72 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:56.744540 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6579dd4-0417-4a6e-b89b-1c54a62d7d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:56.744819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6579dd4-0417-4a6e-b89b-1c54a62d7d72 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:56.745556 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6579dd4-0417-4a6e-b89b-1c54a62d7d72 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:56.747113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6579dd4-0417-4a6e-b89b-1c54a62d7d72 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:56.777808 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 25/46] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jul 12 13:27:56 2022] POST /identity/v3/projects => generated 312 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:58.510695 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92730b04-a8e5-4bf6-b96f-6b3d60957c7d None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:58.511576 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92730b04-a8e5-4bf6-b96f-6b3d60957c7d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:58.512001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92730b04-a8e5-4bf6-b96f-6b3d60957c7d None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:58.513809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 22/47] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:27:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:27:58.520016 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627624f-9043-4bce-8f02-a58ff56af162 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:58.520452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627624f-9043-4bce-8f02-a58ff56af162 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:58.520593 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627624f-9043-4bce-8f02-a58ff56af162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:58.538240 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-e627624f-9043-4bce-8f02-a58ff56af162 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:58.539785 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e627624f-9043-4bce-8f02-a58ff56af162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:58.574731 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:58.624341 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 26/48] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:27:58.648128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92730b04-a8e5-4bf6-b96f-6b3d60957c7d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:58.756293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92730b04-a8e5-4bf6-b96f-6b3d60957c7d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:58.789761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e4634-0b52-43c8-9367-5b069c023496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:58.790500 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e4634-0b52-43c8-9367-5b069c023496 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:58.790787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e4634-0b52-43c8-9367-5b069c023496 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:58.791066 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e4634-0b52-43c8-9367-5b069c023496 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:58.801595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e4634-0b52-43c8-9367-5b069c023496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:27:58.808245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e623824-1cd4-4544-beb1-1c56d21bc351 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:58.808545 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e623824-1cd4-4544-beb1-1c56d21bc351 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:58.808827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e623824-1cd4-4544-beb1-1c56d21bc351 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:58.809313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-921e4634-0b52-43c8-9367-5b069c023496 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:27:58.809313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:27:58.809313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:27:58.809313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:27:58.809313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:27:58.809313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:27:58.809313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:27:58.809313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:27:58.809313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:27:58.809313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:27:58.809313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:27:58.809313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:27:58.809313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:27:58.809313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:27:58.809313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:27:58.809313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:27:58.810218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:27:58.810218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:27:58.810218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:27:58.810218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:27:58.810218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:27:58.810218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:27:58.810218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:27:58.810218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:27:58.810218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:27:58.810218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:27:58.810218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:27:58.810218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:27:58.810218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:27:58.810218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:27:58.810218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:27:58.810218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:27:58.810963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:27:58.810963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:27:58.810963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:27:58.810963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:27:58.810963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:27:58.810963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:27:58.810963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:27:58.810963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:27:58.810963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:27:58.810963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:27:58.810963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:27:58.810963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:27:58.811524 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 27/49] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:27:58.811595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 23/50] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:27:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:27:58.817075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1697e72-99c7-497e-933f-4b5a5175c3b2 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:58.817193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e4634-0b52-43c8-9367-5b069c023496 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:58.817363 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1697e72-99c7-497e-933f-4b5a5175c3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:58.817644 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1697e72-99c7-497e-933f-4b5a5175c3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:58.834114 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b1697e72-99c7-497e-933f-4b5a5175c3b2 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:58.835304 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b1697e72-99c7-497e-933f-4b5a5175c3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:58.870018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:58.905745 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 28/51] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:27:58.920991 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e4634-0b52-43c8-9367-5b069c023496 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:58.929432 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1697e72-99c7-497e-933f-4b5a5175c3b2 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:59.033428 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:59.042797 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1697e72-99c7-497e-933f-4b5a5175c3b2 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:59.063096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485f349-bad7-493a-bae0-58bdb2e4bf33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:59.063780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485f349-bad7-493a-bae0-58bdb2e4bf33 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:59.064067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485f349-bad7-493a-bae0-58bdb2e4bf33 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:59.064350 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485f349-bad7-493a-bae0-58bdb2e4bf33 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:59.065106 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485f349-bad7-493a-bae0-58bdb2e4bf33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:59.065426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485f349-bad7-493a-bae0-58bdb2e4bf33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:59.066864 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485f349-bad7-493a-bae0-58bdb2e4bf33 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:59.075686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 24/52] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:27:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:27:59.081180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485f349-bad7-493a-bae0-58bdb2e4bf33 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:59.082801 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f52bac-86bc-4805-922e-6e6bed59a529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:59.083852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f52bac-86bc-4805-922e-6e6bed59a529 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:59.084368 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f52bac-86bc-4805-922e-6e6bed59a529 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:59.084745 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f52bac-86bc-4805-922e-6e6bed59a529 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:59.086035 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f52bac-86bc-4805-922e-6e6bed59a529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:59.087471 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f52bac-86bc-4805-922e-6e6bed59a529 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:59.089118 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 29/53] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:27:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:27:59.094527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f52bac-86bc-4805-922e-6e6bed59a529 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:59.181405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485f349-bad7-493a-bae0-58bdb2e4bf33 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:59.194142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f52bac-86bc-4805-922e-6e6bed59a529 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:59.194533 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:59.215342 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc4b69-5e18-4a60-a3b7-3cd1497f0469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:59.216254 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc4b69-5e18-4a60-a3b7-3cd1497f0469 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:59.216548 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc4b69-5e18-4a60-a3b7-3cd1497f0469 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:59.216827 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc4b69-5e18-4a60-a3b7-3cd1497f0469 None admin] PATH_INFO: `/v3/users/0ed92bd32dc24cfabfcf0a70a4518dcf` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:59.218311 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc4b69-5e18-4a60-a3b7-3cd1497f0469 None admin] RBAC: Authorizing `identity:get_user(user_id=0ed92bd32dc24cfabfcf0a70a4518dcf)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:59.219779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc4b69-5e18-4a60-a3b7-3cd1497f0469 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:59.221946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 25/54] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:27:59 2022] GET /identity/v3/users/0ed92bd32dc24cfabfcf0a70a4518dcf => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:27:59.227234 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc4b69-5e18-4a60-a3b7-3cd1497f0469 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:59.228104 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b89cb7-e41d-435b-abea-a9a99df34d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:59.228776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b89cb7-e41d-435b-abea-a9a99df34d80 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:59.229062 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b89cb7-e41d-435b-abea-a9a99df34d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:59.229342 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b89cb7-e41d-435b-abea-a9a99df34d80 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:59.230078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b89cb7-e41d-435b-abea-a9a99df34d80 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:59.231584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b89cb7-e41d-435b-abea-a9a99df34d80 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:59.257088 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 30/55] 10.222.0.52 () {60 vars in 1140 bytes} [Tue Jul 12 13:27:59 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:27:59.343390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc4b69-5e18-4a60-a3b7-3cd1497f0469 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:59.379623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed68f6-1b57-4fc2-8192-5a9b15469899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:59.380432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed68f6-1b57-4fc2-8192-5a9b15469899 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:59.380724 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed68f6-1b57-4fc2-8192-5a9b15469899 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:59.381010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed68f6-1b57-4fc2-8192-5a9b15469899 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:59.382332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed68f6-1b57-4fc2-8192-5a9b15469899 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:59.383807 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed68f6-1b57-4fc2-8192-5a9b15469899 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:59.385485 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 26/56] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:27:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:27:59.391464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b89cb7-e41d-435b-abea-a9a99df34d80 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:59.409716 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:59.502232 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b89cb7-e41d-435b-abea-a9a99df34d80 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:59.536592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b849a8-5206-4eb1-9831-3ee16a9b7a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:59.537283 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b849a8-5206-4eb1-9831-3ee16a9b7a69 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:59.537571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b849a8-5206-4eb1-9831-3ee16a9b7a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:59.537849 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b849a8-5206-4eb1-9831-3ee16a9b7a69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:59.538733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b849a8-5206-4eb1-9831-3ee16a9b7a69 None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.domain.id=default, user.id=0ed92bd32dc24cfabfcf0a70a4518dcf) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:59.539045 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b849a8-5206-4eb1-9831-3ee16a9b7a69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:59.540513 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b849a8-5206-4eb1-9831-3ee16a9b7a69 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:59.547358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 31/57] 10.222.0.52 () {56 vars in 1322 bytes} [Tue Jul 12 13:27:59 2022] GET /identity/v3/role_assignments?user.id=0ed92bd32dc24cfabfcf0a70a4518dcf&scope.domain.id=default&role.id=864a6966630544e8bd68bcafba905c6f => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:01.272437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3137898-20ef-4e91-8eda-0ed367b67795 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:01.273093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3137898-20ef-4e91-8eda-0ed367b67795 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:01.273093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3137898-20ef-4e91-8eda-0ed367b67795 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:01.274540 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 27/58] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:01.279644 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dace5-9696-431b-87df-d0d422d99dd0 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:01.280025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dace5-9696-431b-87df-d0d422d99dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:01.280211 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dace5-9696-431b-87df-d0d422d99dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:01.296907 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-6a9dace5-9696-431b-87df-d0d422d99dd0 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:01.298164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6a9dace5-9696-431b-87df-d0d422d99dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:01.363585 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 32/59] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:01.388769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3137898-20ef-4e91-8eda-0ed367b67795 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:01.438021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:01.491087 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3137898-20ef-4e91-8eda-0ed367b67795 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:01.525371 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f22325-d67c-4511-92a9-093444f8cb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:01.526124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f22325-d67c-4511-92a9-093444f8cb28 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:01.526414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f22325-d67c-4511-92a9-093444f8cb28 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:01.526693 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f22325-d67c-4511-92a9-093444f8cb28 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:01.527884 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f22325-d67c-4511-92a9-093444f8cb28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:01.529344 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f22325-d67c-4511-92a9-093444f8cb28 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:01.530993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 28/60] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:28:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:01.536666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dace5-9696-431b-87df-d0d422d99dd0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:01.544802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:01.645523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dace5-9696-431b-87df-d0d422d99dd0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:01.679962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1b1af-0b1d-417c-9854-5182efb4875a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:01.680652 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1b1af-0b1d-417c-9854-5182efb4875a None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:01.680944 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1b1af-0b1d-417c-9854-5182efb4875a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:01.681227 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1b1af-0b1d-417c-9854-5182efb4875a None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:01.681993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1b1af-0b1d-417c-9854-5182efb4875a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:01.683483 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1b1af-0b1d-417c-9854-5182efb4875a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:01.709443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 33/61] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:28:01 2022] POST /identity/v3/users => generated 275 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:04.819247 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8951f9-55d5-42fd-aba9-30079d4c6523 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:04.820448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8951f9-55d5-42fd-aba9-30079d4c6523 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:04.820953 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8951f9-55d5-42fd-aba9-30079d4c6523 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:04.822559 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 29/62] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:04.827914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7240ff-d1c9-43cb-9ec0-e33ffa53ad38 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:04.828763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7240ff-d1c9-43cb-9ec0-e33ffa53ad38 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:04.829783 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7240ff-d1c9-43cb-9ec0-e33ffa53ad38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:04.852214 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-2f7240ff-d1c9-43cb-9ec0-e33ffa53ad38 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:04.853707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2f7240ff-d1c9-43cb-9ec0-e33ffa53ad38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:04.942020 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:04.958113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 34/63] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:04.974685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0609554-f722-414e-9a69-3663fb02a380 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:04.974978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0609554-f722-414e-9a69-3663fb02a380 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:04.975265 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0609554-f722-414e-9a69-3663fb02a380 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:04.976832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 30/64] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:04.981834 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e3cc4-5ce9-4aa9-ace9-7e56e43773c6 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:04.982164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e3cc4-5ce9-4aa9-ace9-7e56e43773c6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:04.982456 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e3cc4-5ce9-4aa9-ace9-7e56e43773c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:04.990547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0609554-f722-414e-9a69-3663fb02a380 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:05.007145 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-092e3cc4-5ce9-4aa9-ace9-7e56e43773c6 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:05.008704 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-092e3cc4-5ce9-4aa9-ace9-7e56e43773c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:05.103996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0609554-f722-414e-9a69-3663fb02a380 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:05.121939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:05.144059 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb131c-e2e3-412c-80b1-5536ddb7a93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:05.145330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb131c-e2e3-412c-80b1-5536ddb7a93c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.145823 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb131c-e2e3-412c-80b1-5536ddb7a93c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.146368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb131c-e2e3-412c-80b1-5536ddb7a93c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.151388 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 35/65] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:05.158794 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb131c-e2e3-412c-80b1-5536ddb7a93c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:05.166923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-ccbb131c-e2e3-412c-80b1-5536ddb7a93c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:05.166923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:05.166923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:05.166923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:05.166923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:05.166923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:05.166923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:05.166923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:05.166923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:05.166923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:05.166923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:05.166923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:05.166923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:05.166923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:05.166923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:05.166923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:05.167771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:05.167771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:05.167771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:05.167771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:05.167771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:05.167771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:05.167771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:05.167771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:05.167771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:05.167771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:05.167771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:05.167771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:05.167771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:05.167771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:05.167771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:05.167771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:05.168547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:05.168547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:05.168547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:05.168547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:05.168547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:05.168547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:05.168547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:05.168547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:05.168547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:05.168547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:05.168547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:05.168547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:05.170897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 31/66] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:28:05.179286 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e3cc4-5ce9-4aa9-ace9-7e56e43773c6 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:05.185942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb131c-e2e3-412c-80b1-5536ddb7a93c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:05.306104 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb131c-e2e3-412c-80b1-5536ddb7a93c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:05.350661 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa798c7-cc65-4baa-8a94-9c151590f2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:05.352509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa798c7-cc65-4baa-8a94-9c151590f2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.352979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa798c7-cc65-4baa-8a94-9c151590f2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.353458 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa798c7-cc65-4baa-8a94-9c151590f2a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.367171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa798c7-cc65-4baa-8a94-9c151590f2a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:05.377836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-2aa798c7-cc65-4baa-8a94-9c151590f2a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:05.377836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:05.377836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:05.377836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:05.377836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:05.377836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:05.377836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:05.377836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:05.377836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:05.377836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:05.377836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:05.377836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:05.377836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:05.377836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:05.377836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:05.377836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:05.379173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:05.379173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:05.379173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:05.379173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:05.379173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:05.379173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:05.379173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:05.379173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:05.379173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:05.379173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:05.379173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:05.379173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:05.379173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:05.379173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:05.379173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:05.379173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:05.380139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:05.380139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:05.380139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:05.380139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:05.380139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:05.380139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:05.380139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:05.380139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:05.380139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:05.380139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:05.380139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:05.380139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:05.382791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 32/67] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:28:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:28:05.383900 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e3cc4-5ce9-4aa9-ace9-7e56e43773c6 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:05.384310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597c157-e14f-4940-9102-4725f35b08fe None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.384606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597c157-e14f-4940-9102-4725f35b08fe None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.384893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597c157-e14f-4940-9102-4725f35b08fe None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.386389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 33/68] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:05.390365 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6ee76-8fe9-475b-b8cb-16995838bd66 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.390365 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6ee76-8fe9-475b-b8cb-16995838bd66 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.390365 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6ee76-8fe9-475b-b8cb-16995838bd66 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.390365 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 34/69] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:05.390624 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6ee76-8fe9-475b-b8cb-16995838bd66 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:05.413052 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:05.427095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639da7aa-2719-4d27-978f-8704713bb043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:05.428054 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639da7aa-2719-4d27-978f-8704713bb043 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.428410 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639da7aa-2719-4d27-978f-8704713bb043 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.428694 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639da7aa-2719-4d27-978f-8704713bb043 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.429445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639da7aa-2719-4d27-978f-8704713bb043 None admin] RBAC: Adding query filter params (name=member) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:05.429761 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639da7aa-2719-4d27-978f-8704713bb043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:05.431209 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639da7aa-2719-4d27-978f-8704713bb043 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:05.436962 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:05.441004 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 36/70] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:05.443305 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fe41c-7754-41c8-a1af-801fc0d4f048 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.443602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fe41c-7754-41c8-a1af-801fc0d4f048 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.443883 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fe41c-7754-41c8-a1af-801fc0d4f048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.461777 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5b7fe41c-7754-41c8-a1af-801fc0d4f048 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:05.463098 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5b7fe41c-7754-41c8-a1af-801fc0d4f048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:05.508761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6ee76-8fe9-475b-b8cb-16995838bd66 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:05.538208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 37/71] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:05.540282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9faeb3-fbf9-4815-bab5-2f03ecd7482e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.541071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9faeb3-fbf9-4815-bab5-2f03ecd7482e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.541823 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9faeb3-fbf9-4815-bab5-2f03ecd7482e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.553950 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9971e6c-23a0-4a69-bbcd-e3b6bca2541c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:05.554784 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9971e6c-23a0-4a69-bbcd-e3b6bca2541c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.555136 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9971e6c-23a0-4a69-bbcd-e3b6bca2541c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.555480 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9971e6c-23a0-4a69-bbcd-e3b6bca2541c None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.556485 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9971e6c-23a0-4a69-bbcd-e3b6bca2541c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:05.556855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9971e6c-23a0-4a69-bbcd-e3b6bca2541c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:05.558476 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9971e6c-23a0-4a69-bbcd-e3b6bca2541c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:05.567833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 35/72] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:05.568655 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9971e6c-23a0-4a69-bbcd-e3b6bca2541c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:05.569468 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8a9faeb3-fbf9-4815-bab5-2f03ecd7482e None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:05.570686 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8a9faeb3-fbf9-4815-bab5-2f03ecd7482e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:05.637331 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 38/73] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:05.638611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9faeb3-fbf9-4815-bab5-2f03ecd7482e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:05.667203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:05.671276 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9971e6c-23a0-4a69-bbcd-e3b6bca2541c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:05.700688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:05.707235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407e297-7e49-40ad-ac07-8df44ef1feca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:05.708121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407e297-7e49-40ad-ac07-8df44ef1feca None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.708411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407e297-7e49-40ad-ac07-8df44ef1feca None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.708693 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407e297-7e49-40ad-ac07-8df44ef1feca None admin] PATH_INFO: `/v3/users/febf11689a9345e18b3809a7135a9b7f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.724194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407e297-7e49-40ad-ac07-8df44ef1feca None admin] RBAC: Authorizing `identity:get_user(user_id=febf11689a9345e18b3809a7135a9b7f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:05.725644 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407e297-7e49-40ad-ac07-8df44ef1feca None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:05.727386 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 36/74] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:05 2022] GET /identity/v3/users/febf11689a9345e18b3809a7135a9b7f => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:05.728682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863f811-f1ee-48e3-8561-ebfee9692f55 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.729001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863f811-f1ee-48e3-8561-ebfee9692f55 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.729278 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863f811-f1ee-48e3-8561-ebfee9692f55 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.730687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 37/75] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:05.734782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863f811-f1ee-48e3-8561-ebfee9692f55 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:05.756693 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9faeb3-fbf9-4815-bab5-2f03ecd7482e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:05.792743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a43d6-8939-48f7-8e1d-08f09b12602f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:05.793572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a43d6-8939-48f7-8e1d-08f09b12602f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.793860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a43d6-8939-48f7-8e1d-08f09b12602f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.794175 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a43d6-8939-48f7-8e1d-08f09b12602f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.804234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a43d6-8939-48f7-8e1d-08f09b12602f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:05.812494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-e92a43d6-8939-48f7-8e1d-08f09b12602f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:05.812494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:05.812494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:05.812494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:05.812494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:05.812494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:05.812494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:05.812494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:05.812494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:05.812494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:05.812494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:05.812494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:05.812494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:05.812494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:05.812494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:05.812494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:05.813896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:05.813896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:05.813896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:05.813896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:05.813896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:05.813896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:05.813896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:05.813896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:05.813896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:05.813896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:05.813896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:05.813896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:05.813896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:05.813896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:05.813896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:05.813896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:05.814687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:05.814687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:05.814687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:05.814687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:05.814687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:05.814687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:05.814687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:05.814687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:05.814687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:05.814687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:05.814687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:05.814687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:05.814687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 39/76] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:28:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:05.815323 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a43d6-8939-48f7-8e1d-08f09b12602f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:05.840806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863f811-f1ee-48e3-8561-ebfee9692f55 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:05.876290 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6628c0-f32c-4148-9570-b1b747f54c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:05.877258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6628c0-f32c-4148-9570-b1b747f54c61 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.877612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6628c0-f32c-4148-9570-b1b747f54c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.877981 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6628c0-f32c-4148-9570-b1b747f54c61 None admin] PATH_INFO: `/v3/users/0ed92bd32dc24cfabfcf0a70a4518dcf` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.879516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6628c0-f32c-4148-9570-b1b747f54c61 None admin] RBAC: Authorizing `identity:get_user(user_id=0ed92bd32dc24cfabfcf0a70a4518dcf)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:05.880975 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6628c0-f32c-4148-9570-b1b747f54c61 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:05.882645 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 38/77] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:05 2022] GET /identity/v3/users/0ed92bd32dc24cfabfcf0a70a4518dcf => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:05.883397 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6628c0-f32c-4148-9570-b1b747f54c61 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:05.910501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:05.916603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a43d6-8939-48f7-8e1d-08f09b12602f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:05.919409 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:05.950641 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f5ca3-2f81-4e41-b063-f6f705214049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:05.951447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f5ca3-2f81-4e41-b063-f6f705214049 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.951728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f5ca3-2f81-4e41-b063-f6f705214049 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.952021 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f5ca3-2f81-4e41-b063-f6f705214049 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.962458 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f5ca3-2f81-4e41-b063-f6f705214049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:05.970276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-a36f5ca3-2f81-4e41-b063-f6f705214049 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:05.970276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:05.970276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:05.970276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:05.970276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:05.970276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:05.970276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:05.970276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:05.970276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:05.970276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:05.970276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:05.970276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:05.970276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:05.970276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:05.970276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:05.970276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:05.971111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:05.971111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:05.971111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:05.971111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:05.971111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:05.971111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:05.971111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:05.971111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:05.971111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:05.971111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:05.971111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:05.971111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:05.971111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:05.971111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:05.971111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:05.971111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:05.971875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:05.971875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:05.971875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:05.971875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:05.971875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:05.971875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:05.971875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:05.971875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:05.971875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:05.971875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:05.971875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:05.971875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:05.971875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 40/78] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:05.972954 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580b128-7656-4320-871a-9ad474f3a4ce None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.972954 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580b128-7656-4320-871a-9ad474f3a4ce None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.973246 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580b128-7656-4320-871a-9ad474f3a4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.988286 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6628c0-f32c-4148-9570-b1b747f54c61 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:05.989371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a580b128-7656-4320-871a-9ad474f3a4ce None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:05.990596 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a580b128-7656-4320-871a-9ad474f3a4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:06.022768 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace39466-a5f1-48b1-887f-1db5c8c81466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:06.023417 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace39466-a5f1-48b1-887f-1db5c8c81466 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:06.023700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace39466-a5f1-48b1-887f-1db5c8c81466 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:06.023975 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace39466-a5f1-48b1-887f-1db5c8c81466 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:06.058452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 41/79] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:06.059486 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580b128-7656-4320-871a-9ad474f3a4ce None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:06.089727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace39466-a5f1-48b1-887f-1db5c8c81466 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:06.091240 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace39466-a5f1-48b1-887f-1db5c8c81466 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:06.115829 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:06.158110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:06.165473 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580b128-7656-4320-871a-9ad474f3a4ce None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:06.190812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 39/80] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:06.191556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace39466-a5f1-48b1-887f-1db5c8c81466 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:06.203128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423334e0-9e1c-42a4-84a0-77faa019c201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:06.203807 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423334e0-9e1c-42a4-84a0-77faa019c201 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:06.204117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423334e0-9e1c-42a4-84a0-77faa019c201 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:06.204379 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423334e0-9e1c-42a4-84a0-77faa019c201 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:06.205128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423334e0-9e1c-42a4-84a0-77faa019c201 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:06.205440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423334e0-9e1c-42a4-84a0-77faa019c201 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:06.206893 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423334e0-9e1c-42a4-84a0-77faa019c201 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:06.215116 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 42/81] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:28:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:06.215898 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423334e0-9e1c-42a4-84a0-77faa019c201 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:06.292637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace39466-a5f1-48b1-887f-1db5c8c81466 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:06.312505 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423334e0-9e1c-42a4-84a0-77faa019c201 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:06.315921 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:06.326881 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6e34e-056e-47d2-bdf7-0d17840afc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:06.327538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6e34e-056e-47d2-bdf7-0d17840afc8e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:06.327823 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6e34e-056e-47d2-bdf7-0d17840afc8e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:06.328102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6e34e-056e-47d2-bdf7-0d17840afc8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:06.345436 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ba21e-a246-4047-a7fa-a9209ad30857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:06.346129 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ba21e-a246-4047-a7fa-a9209ad30857 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:06.346416 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ba21e-a246-4047-a7fa-a9209ad30857 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:06.346696 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ba21e-a246-4047-a7fa-a9209ad30857 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:06.347437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ba21e-a246-4047-a7fa-a9209ad30857 None admin] RBAC: Adding query filter params (name=member) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:06.347745 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ba21e-a246-4047-a7fa-a9209ad30857 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:06.349144 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ba21e-a246-4047-a7fa-a9209ad30857 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:06.357103 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 43/82] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:06.357887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ba21e-a246-4047-a7fa-a9209ad30857 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:06.379170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:06.396969 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6e34e-056e-47d2-bdf7-0d17840afc8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:06.398758 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6e34e-056e-47d2-bdf7-0d17840afc8e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:06.458702 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ba21e-a246-4047-a7fa-a9209ad30857 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:06.492810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c042f-93bb-4800-b45b-0c764251a962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:06.493610 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c042f-93bb-4800-b45b-0c764251a962 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:06.493895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c042f-93bb-4800-b45b-0c764251a962 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:06.494203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c042f-93bb-4800-b45b-0c764251a962 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:06.495371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c042f-93bb-4800-b45b-0c764251a962 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:06.495783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 40/83] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:06.496629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6e34e-056e-47d2-bdf7-0d17840afc8e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:06.496932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c042f-93bb-4800-b45b-0c764251a962 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:06.498912 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 44/84] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:28:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:06.499435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c042f-93bb-4800-b45b-0c764251a962 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:06.521669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:06.587785 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:06.598414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6e34e-056e-47d2-bdf7-0d17840afc8e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:06.603110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c042f-93bb-4800-b45b-0c764251a962 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:06.633552 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a49e8-5258-457b-bb10-fd640919d48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:06.634454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a49e8-5258-457b-bb10-fd640919d48e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:06.634743 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a49e8-5258-457b-bb10-fd640919d48e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:06.635033 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a49e8-5258-457b-bb10-fd640919d48e None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:06.636200 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a49e8-5258-457b-bb10-fd640919d48e None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:06.637665 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a49e8-5258-457b-bb10-fd640919d48e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:06.637768 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44531e22-c995-479a-96a3-4b65c900c259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:06.638600 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44531e22-c995-479a-96a3-4b65c900c259 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:06.638885 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44531e22-c995-479a-96a3-4b65c900c259 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:06.639168 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44531e22-c995-479a-96a3-4b65c900c259 None admin] PATH_INFO: `/v3/users/febf11689a9345e18b3809a7135a9b7f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:06.639417 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 41/85] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:06 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:06.641023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44531e22-c995-479a-96a3-4b65c900c259 None admin] RBAC: Authorizing `identity:get_user(user_id=febf11689a9345e18b3809a7135a9b7f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:06.642465 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44531e22-c995-479a-96a3-4b65c900c259 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:06.648651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 45/86] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:06 2022] GET /identity/v3/users/febf11689a9345e18b3809a7135a9b7f => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:06.648651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44531e22-c995-479a-96a3-4b65c900c259 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:06.648843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a49e8-5258-457b-bb10-fd640919d48e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:06.737459 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:06.746732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a49e8-5258-457b-bb10-fd640919d48e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:06.748981 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44531e22-c995-479a-96a3-4b65c900c259 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:06.782879 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00149f1c-a145-4892-b86b-a66cedf3c660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:06.783436 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19d3bf-41c7-44c8-901f-b9f23923a0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:06.783769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00149f1c-a145-4892-b86b-a66cedf3c660 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:06.784055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00149f1c-a145-4892-b86b-a66cedf3c660 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:06.784306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19d3bf-41c7-44c8-901f-b9f23923a0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:06.784404 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00149f1c-a145-4892-b86b-a66cedf3c660 None admin] PATH_INFO: `/v3/users/febf11689a9345e18b3809a7135a9b7f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:06.784596 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19d3bf-41c7-44c8-901f-b9f23923a0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:06.784880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19d3bf-41c7-44c8-901f-b9f23923a0d7 None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:06.785544 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00149f1c-a145-4892-b86b-a66cedf3c660 None admin] RBAC: Authorizing `identity:get_user(user_id=febf11689a9345e18b3809a7135a9b7f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:06.786097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19d3bf-41c7-44c8-901f-b9f23923a0d7 None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:06.787051 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00149f1c-a145-4892-b86b-a66cedf3c660 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:06.787556 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19d3bf-41c7-44c8-901f-b9f23923a0d7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:06.788691 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 42/87] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:06 2022] GET /identity/v3/users/febf11689a9345e18b3809a7135a9b7f => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:06.789435 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00149f1c-a145-4892-b86b-a66cedf3c660 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:06.789880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 46/88] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:06 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:06.790670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19d3bf-41c7-44c8-901f-b9f23923a0d7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:06.816373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:06.892824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19d3bf-41c7-44c8-901f-b9f23923a0d7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:06.902318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00149f1c-a145-4892-b86b-a66cedf3c660 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:06.925491 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a4b4a-fe0d-49a3-be32-ee5c7cbddeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:06.926168 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a4b4a-fe0d-49a3-be32-ee5c7cbddeb0 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:06.926456 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a4b4a-fe0d-49a3-be32-ee5c7cbddeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:06.926736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a4b4a-fe0d-49a3-be32-ee5c7cbddeb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:06.927553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a4b4a-fe0d-49a3-be32-ee5c7cbddeb0 None admin] RBAC: Adding query filter params (role.id=ed842da869724411a5b355cd12f1c5c8, scope.project.id=abaed0828ac34adb801becefa86d65b3, user.id=febf11689a9345e18b3809a7135a9b7f) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:06.927867 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a4b4a-fe0d-49a3-be32-ee5c7cbddeb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:06.929311 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a4b4a-fe0d-49a3-be32-ee5c7cbddeb0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:06.934606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 47/89] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:06 2022] GET /identity/v3/role_assignments?user.id=febf11689a9345e18b3809a7135a9b7f&scope.project.id=abaed0828ac34adb801becefa86d65b3&role.id=ed842da869724411a5b355cd12f1c5c8 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:06.946247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a4b4a-fe0d-49a3-be32-ee5c7cbddeb0 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:06.946366 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08844fe9-c18f-441e-862c-729010ac5c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:06.946366 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08844fe9-c18f-441e-862c-729010ac5c97 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:06.946366 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08844fe9-c18f-441e-862c-729010ac5c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:06.946366 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08844fe9-c18f-441e-862c-729010ac5c97 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:06.946657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08844fe9-c18f-441e-862c-729010ac5c97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:06.946657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08844fe9-c18f-441e-862c-729010ac5c97 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:06.971182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 43/90] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:28:06 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:06.972219 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08844fe9-c18f-441e-862c-729010ac5c97 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:07.002792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:07.079869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:07.096383 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a4b4a-fe0d-49a3-be32-ee5c7cbddeb0 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:07.127248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08844fe9-c18f-441e-862c-729010ac5c97 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:07.145318 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10585805-adf9-4c69-bdd6-b6f3c52aed7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:07.146012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10585805-adf9-4c69-bdd6-b6f3c52aed7e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:07.146308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10585805-adf9-4c69-bdd6-b6f3c52aed7e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:07.146594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10585805-adf9-4c69-bdd6-b6f3c52aed7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:07.169590 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9727868-e7bb-4d1b-b13d-6c2d66ff8763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:07.170319 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9727868-e7bb-4d1b-b13d-6c2d66ff8763 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:07.170612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9727868-e7bb-4d1b-b13d-6c2d66ff8763 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:07.170944 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9727868-e7bb-4d1b-b13d-6c2d66ff8763 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:07.233988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10585805-adf9-4c69-bdd6-b6f3c52aed7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:07.235589 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10585805-adf9-4c69-bdd6-b6f3c52aed7e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:07.257892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9727868-e7bb-4d1b-b13d-6c2d66ff8763 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:07.259682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9727868-e7bb-4d1b-b13d-6c2d66ff8763 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:07.281886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:07.345692 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:07.366136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 48/91] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:07.367048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10585805-adf9-4c69-bdd6-b6f3c52aed7e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:07.392640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 44/92] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:07.394071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9727868-e7bb-4d1b-b13d-6c2d66ff8763 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:07.492998 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10585805-adf9-4c69-bdd6-b6f3c52aed7e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:07.502475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9727868-e7bb-4d1b-b13d-6c2d66ff8763 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:07.532561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c3639-a130-49e2-877c-9dbbc9f65ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:07.533321 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c3639-a130-49e2-877c-9dbbc9f65ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:07.533616 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c3639-a130-49e2-877c-9dbbc9f65ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:07.533897 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c3639-a130-49e2-877c-9dbbc9f65ff9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:07.534795 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c3639-a130-49e2-877c-9dbbc9f65ff9 None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=abaed0828ac34adb801becefa86d65b3, user.id=0ed92bd32dc24cfabfcf0a70a4518dcf) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:07.535117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c3639-a130-49e2-877c-9dbbc9f65ff9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:07.536628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c3639-a130-49e2-877c-9dbbc9f65ff9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:07.541294 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f33aaa-7c3b-43dd-ac54-b150e00aee6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:07.542478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f33aaa-7c3b-43dd-ac54-b150e00aee6b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:07.542961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f33aaa-7c3b-43dd-ac54-b150e00aee6b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:07.543062 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 49/93] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:07 2022] GET /identity/v3/role_assignments?user.id=0ed92bd32dc24cfabfcf0a70a4518dcf&scope.project.id=abaed0828ac34adb801becefa86d65b3&role.id=864a6966630544e8bd68bcafba905c6f => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:07.543516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f33aaa-7c3b-43dd-ac54-b150e00aee6b None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:07.543938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c3639-a130-49e2-877c-9dbbc9f65ff9 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:07.545557 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f33aaa-7c3b-43dd-ac54-b150e00aee6b None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:07.548002 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f33aaa-7c3b-43dd-ac54-b150e00aee6b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:07.550922 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 45/94] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:07 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:07.556598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:07.557943 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f33aaa-7c3b-43dd-ac54-b150e00aee6b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:07.603834 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:07.665999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c3639-a130-49e2-877c-9dbbc9f65ff9 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:07.738959 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7f16b-9ec6-4f5b-8955-7ab7c9c12e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:07.739890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7f16b-9ec6-4f5b-8955-7ab7c9c12e98 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:07.740179 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7f16b-9ec6-4f5b-8955-7ab7c9c12e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:07.740458 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7f16b-9ec6-4f5b-8955-7ab7c9c12e98 None admin] PATH_INFO: `/v3/projects/b9b42dc83e944315b0439a61f932334f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:07.741746 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7f16b-9ec6-4f5b-8955-7ab7c9c12e98 None admin] RBAC: Authorizing `identity:get_project(project_id=b9b42dc83e944315b0439a61f932334f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:07.743330 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7f16b-9ec6-4f5b-8955-7ab7c9c12e98 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:07.751526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f33aaa-7c3b-43dd-ac54-b150e00aee6b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:07.755705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 50/95] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:07 2022] GET /identity/v3/projects/b9b42dc83e944315b0439a61f932334f => generated 312 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:07.764718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7f16b-9ec6-4f5b-8955-7ab7c9c12e98 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:07.835503 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b3c56-4a51-461a-a19d-609cdfd87c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:07.836082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b3c56-4a51-461a-a19d-609cdfd87c22 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:07.836372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b3c56-4a51-461a-a19d-609cdfd87c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:07.836650 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b3c56-4a51-461a-a19d-609cdfd87c22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:07.838179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b3c56-4a51-461a-a19d-609cdfd87c22 None admin] RBAC: Adding query filter params (role.id=dbe30950eded4c9ab2bcc0f7545b9bae, scope.project.id=abaed0828ac34adb801becefa86d65b3, user.id=febf11689a9345e18b3809a7135a9b7f) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:07.838505 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b3c56-4a51-461a-a19d-609cdfd87c22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:07.839978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b3c56-4a51-461a-a19d-609cdfd87c22 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:07.867876 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 46/96] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:07 2022] GET /identity/v3/role_assignments?user.id=febf11689a9345e18b3809a7135a9b7f&scope.project.id=abaed0828ac34adb801becefa86d65b3&role.id=dbe30950eded4c9ab2bcc0f7545b9bae => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:07.954333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7f16b-9ec6-4f5b-8955-7ab7c9c12e98 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:07.974949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:07.994937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc511c12-c244-46aa-a21d-316fafd06890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:07.995628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc511c12-c244-46aa-a21d-316fafd06890 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:07.995916 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc511c12-c244-46aa-a21d-316fafd06890 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:07.996199 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc511c12-c244-46aa-a21d-316fafd06890 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:07.997058 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc511c12-c244-46aa-a21d-316fafd06890 None admin] RBAC: Adding query filter params (role.id=ed842da869724411a5b355cd12f1c5c8, scope.project.id=b9b42dc83e944315b0439a61f932334f, user.id=febf11689a9345e18b3809a7135a9b7f) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:07.997376 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc511c12-c244-46aa-a21d-316fafd06890 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:07.998901 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc511c12-c244-46aa-a21d-316fafd06890 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:08.005532 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 51/97] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:07 2022] GET /identity/v3/role_assignments?user.id=febf11689a9345e18b3809a7135a9b7f&scope.project.id=b9b42dc83e944315b0439a61f932334f&role.id=ed842da869724411a5b355cd12f1c5c8 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:09.706699 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418260e-ef19-4683-ac1c-9bdb2c47ffb3 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.707443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418260e-ef19-4683-ac1c-9bdb2c47ffb3 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.707443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418260e-ef19-4683-ac1c-9bdb2c47ffb3 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.708974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 47/98] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:09.715262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4d878-a6e7-4735-8475-9712b9ffd074 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.716194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4d878-a6e7-4735-8475-9712b9ffd074 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.716653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4d878-a6e7-4735-8475-9712b9ffd074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.740180 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a9f4d878-a6e7-4735-8475-9712b9ffd074 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:09.741392 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a9f4d878-a6e7-4735-8475-9712b9ffd074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:09.827829 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 52/99] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:09.868099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418260e-ef19-4683-ac1c-9bdb2c47ffb3 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.987938 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:10.108000 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418260e-ef19-4683-ac1c-9bdb2c47ffb3 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.121702 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1273f-e1f8-46c5-bffc-6a5056348498 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.122049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1273f-e1f8-46c5-bffc-6a5056348498 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.122312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1273f-e1f8-46c5-bffc-6a5056348498 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.123845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 53/100] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:10.131210 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efaa9a-c9e9-4b8a-8143-5a9a02fdb857 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.131501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efaa9a-c9e9-4b8a-8143-5a9a02fdb857 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.131785 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efaa9a-c9e9-4b8a-8143-5a9a02fdb857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.174966 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b5efaa9a-c9e9-4b8a-8143-5a9a02fdb857 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:10.176170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b5efaa9a-c9e9-4b8a-8143-5a9a02fdb857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:10.200530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a280e6-c674-415a-8e7b-c21dd11032ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.200530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a280e6-c674-415a-8e7b-c21dd11032ac None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.200530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a280e6-c674-415a-8e7b-c21dd11032ac None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.200530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a280e6-c674-415a-8e7b-c21dd11032ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.216501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a280e6-c674-415a-8e7b-c21dd11032ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:10.235215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-29a280e6-c674-415a-8e7b-c21dd11032ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:10.235215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:10.235215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:10.235215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:10.235215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:10.235215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:10.235215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:10.235215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:10.235215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:10.235215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:10.235215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:10.235215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:10.235215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:10.235215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:10.235215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:10.235215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:10.236074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:10.236074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:10.236074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:10.236074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:10.236074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:10.236074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:10.236074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:10.236074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:10.236074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:10.236074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:10.236074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:10.236074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:10.236074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:10.236074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:10.236074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:10.236074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:10.236813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:10.236813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:10.236813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:10.236813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:10.236813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:10.236813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:10.236813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:10.236813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:10.236813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:10.236813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:10.236813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:10.236813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:10.239671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 48/101] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:10.245639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a280e6-c674-415a-8e7b-c21dd11032ac None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.361906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 54/102] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 12 13:28:10.384544 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a280e6-c674-415a-8e7b-c21dd11032ac None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.421495 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:10.424968 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac67ee3-2f99-40ef-b109-889bb36c973e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.425909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac67ee3-2f99-40ef-b109-889bb36c973e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.428444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac67ee3-2f99-40ef-b109-889bb36c973e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.428944 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac67ee3-2f99-40ef-b109-889bb36c973e None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.429912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac67ee3-2f99-40ef-b109-889bb36c973e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:10.430472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac67ee3-2f99-40ef-b109-889bb36c973e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:10.432213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac67ee3-2f99-40ef-b109-889bb36c973e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:10.435083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efaa9a-c9e9-4b8a-8143-5a9a02fdb857 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.442328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 49/103] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:10.452611 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac67ee3-2f99-40ef-b109-889bb36c973e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.545938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:10.599792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac67ee3-2f99-40ef-b109-889bb36c973e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.661135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efaa9a-c9e9-4b8a-8143-5a9a02fdb857 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.696579 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e6b23-ba16-4601-b965-b4ce329377ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.698127 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e6b23-ba16-4601-b965-b4ce329377ee None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.698642 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e6b23-ba16-4601-b965-b4ce329377ee None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.699154 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e6b23-ba16-4601-b965-b4ce329377ee None admin] PATH_INFO: `/v3/users/d9aec001af964278b6c0c22f50eddf75` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.751543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e6b23-ba16-4601-b965-b4ce329377ee None admin] RBAC: Authorizing `identity:get_user(user_id=d9aec001af964278b6c0c22f50eddf75)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:10.754521 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e6b23-ba16-4601-b965-b4ce329377ee None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:10.757853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 50/104] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:10 2022] GET /identity/v3/users/d9aec001af964278b6c0c22f50eddf75 => generated 289 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:10.768187 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7f9cd-924c-4d66-8388-7bdb6c855d63 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.768681 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7f9cd-924c-4d66-8388-7bdb6c855d63 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.772921 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7f9cd-924c-4d66-8388-7bdb6c855d63 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.775376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 51/105] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:10 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:10.776632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7f9cd-924c-4d66-8388-7bdb6c855d63 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.791768 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e1fbe-eaed-4187-a486-03bda64d2670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.793081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e1fbe-eaed-4187-a486-03bda64d2670 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.793581 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e1fbe-eaed-4187-a486-03bda64d2670 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.794154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e1fbe-eaed-4187-a486-03bda64d2670 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.796311 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e1fbe-eaed-4187-a486-03bda64d2670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:10.799555 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e1fbe-eaed-4187-a486-03bda64d2670 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:10.802604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 55/106] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:28:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:10.804710 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d4b23-0254-4558-adff-ae362c609756 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.805188 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d4b23-0254-4558-adff-ae362c609756 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.805658 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d4b23-0254-4558-adff-ae362c609756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.861341 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b35d4b23-0254-4558-adff-ae362c609756 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:10.863401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b35d4b23-0254-4558-adff-ae362c609756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:10.871670 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:10.951823 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7f9cd-924c-4d66-8388-7bdb6c855d63 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.989188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0342e7-756b-4a2d-9624-48032cf18680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.990252 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0342e7-756b-4a2d-9624-48032cf18680 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.990740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0342e7-756b-4a2d-9624-48032cf18680 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.991230 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0342e7-756b-4a2d-9624-48032cf18680 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.996983 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 56/107] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:10.998543 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d4b23-0254-4558-adff-ae362c609756 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:11.065932 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0342e7-756b-4a2d-9624-48032cf18680 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:11.067749 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0342e7-756b-4a2d-9624-48032cf18680 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:11.078017 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:11.098046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:11.171034 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d4b23-0254-4558-adff-ae362c609756 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:11.174626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 52/108] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:11.179901 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef438e5-b0a4-4696-b453-39b95138894c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.181347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef438e5-b0a4-4696-b453-39b95138894c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.181870 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef438e5-b0a4-4696-b453-39b95138894c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.185924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 53/109] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:11 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:11.187153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef438e5-b0a4-4696-b453-39b95138894c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:11.211815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf79b9-600e-4728-bf37-3535fd26e8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:11.212487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf79b9-600e-4728-bf37-3535fd26e8f9 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.212774 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf79b9-600e-4728-bf37-3535fd26e8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.213055 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf79b9-600e-4728-bf37-3535fd26e8f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.213765 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf79b9-600e-4728-bf37-3535fd26e8f9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:11.215283 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf79b9-600e-4728-bf37-3535fd26e8f9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:11.243010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 57/110] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jul 12 13:28:10 2022] POST /identity/v3/projects => generated 302 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:11.244123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e53f8c-131d-4be7-a27b-6e13668d0b41 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.244574 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e53f8c-131d-4be7-a27b-6e13668d0b41 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.244991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e53f8c-131d-4be7-a27b-6e13668d0b41 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.246237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 58/111] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:11.251614 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e53f8c-131d-4be7-a27b-6e13668d0b41 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:11.331646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef438e5-b0a4-4696-b453-39b95138894c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:11.365714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:11.371677 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:11.381134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e53f8c-131d-4be7-a27b-6e13668d0b41 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:11.383566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be45e06-4414-4b3b-9eca-58b7872614e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:11.385597 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be45e06-4414-4b3b-9eca-58b7872614e7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.386120 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be45e06-4414-4b3b-9eca-58b7872614e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.386577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be45e06-4414-4b3b-9eca-58b7872614e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.401990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be45e06-4414-4b3b-9eca-58b7872614e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:11.412854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-5be45e06-4414-4b3b-9eca-58b7872614e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:11.412854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:11.412854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:11.412854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:11.412854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:11.412854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:11.412854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:11.412854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:11.412854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:11.412854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:11.412854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:11.412854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:11.412854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:11.412854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:11.412854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:11.412854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:11.413736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:11.413736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:11.413736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:11.413736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:11.413736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:11.413736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:11.413736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:11.413736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:11.413736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:11.413736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:11.413736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:11.413736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:11.413736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:11.413736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:11.413736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:11.413736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:11.414526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:11.414526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:11.414526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:11.414526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:11.414526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:11.414526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:11.414526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:11.414526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:11.414526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:11.414526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:11.414526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:11.414526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:11.414526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 54/112] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:11.415694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025891d0-4bd8-4a87-ac4c-ed04c04a1ba4 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.416183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025891d0-4bd8-4a87-ac4c-ed04c04a1ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.416603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025891d0-4bd8-4a87-ac4c-ed04c04a1ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.429781 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1905f-4da3-4189-8b3b-b4be44202301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:11.431190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1905f-4da3-4189-8b3b-b4be44202301 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.431627 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1905f-4da3-4189-8b3b-b4be44202301 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.432047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1905f-4da3-4189-8b3b-b4be44202301 None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.433401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b1905f-4da3-4189-8b3b-b4be44202301 None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:11.435168 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b1905f-4da3-4189-8b3b-b4be44202301 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:11.435283 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-025891d0-4bd8-4a87-ac4c-ed04c04a1ba4 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:11.436596 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-025891d0-4bd8-4a87-ac4c-ed04c04a1ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:11.437729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 59/113] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:11 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:11.439382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aee396-0409-42af-a06d-8e1abd17157b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.439937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aee396-0409-42af-a06d-8e1abd17157b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.440346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aee396-0409-42af-a06d-8e1abd17157b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.460714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-41aee396-0409-42af-a06d-8e1abd17157b None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:11.462135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-41aee396-0409-42af-a06d-8e1abd17157b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:11.521801 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 55/114] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:11.523283 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025891d0-4bd8-4a87-ac4c-ed04c04a1ba4 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:11.542316 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 60/115] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:11.544110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aee396-0409-42af-a06d-8e1abd17157b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:11.673317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:11.673560 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025891d0-4bd8-4a87-ac4c-ed04c04a1ba4 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:11.691614 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aee396-0409-42af-a06d-8e1abd17157b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:11.697904 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:11.723858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b99adb-c79d-4410-9fcc-9b2f6bdd41ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:11.724783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b99adb-c79d-4410-9fcc-9b2f6bdd41ad None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.725388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b99adb-c79d-4410-9fcc-9b2f6bdd41ad None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.725808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b99adb-c79d-4410-9fcc-9b2f6bdd41ad None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.727537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b99adb-c79d-4410-9fcc-9b2f6bdd41ad None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:11.727996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b99adb-c79d-4410-9fcc-9b2f6bdd41ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:11.730014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b99adb-c79d-4410-9fcc-9b2f6bdd41ad None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:11.739475 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57cad0-5778-4137-bf91-c0e47e9c86c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:11.740284 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57cad0-5778-4137-bf91-c0e47e9c86c7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.740679 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57cad0-5778-4137-bf91-c0e47e9c86c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.741046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 56/116] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:11.742083 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b99adb-c79d-4410-9fcc-9b2f6bdd41ad None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:11.745820 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57cad0-5778-4137-bf91-c0e47e9c86c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.747896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57cad0-5778-4137-bf91-c0e47e9c86c7 None admin] RBAC: Adding query filter params (role.id=3d88c071954947f99ede3e31fb58494a, scope.project.id=abaed0828ac34adb801becefa86d65b3, user.id=d9aec001af964278b6c0c22f50eddf75) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:11.748355 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57cad0-5778-4137-bf91-c0e47e9c86c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:11.749999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57cad0-5778-4137-bf91-c0e47e9c86c7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:11.760744 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 61/117] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:11 2022] GET /identity/v3/role_assignments?user.id=d9aec001af964278b6c0c22f50eddf75&scope.project.id=abaed0828ac34adb801becefa86d65b3&role.id=3d88c071954947f99ede3e31fb58494a => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:11.762073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57cad0-5778-4137-bf91-c0e47e9c86c7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:11.877053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b99adb-c79d-4410-9fcc-9b2f6bdd41ad None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:11.914953 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c1193-7139-49a1-94e5-c822dc07d0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:11.915944 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c1193-7139-49a1-94e5-c822dc07d0fa None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.916383 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c1193-7139-49a1-94e5-c822dc07d0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.916761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c1193-7139-49a1-94e5-c822dc07d0fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.917526 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57cad0-5778-4137-bf91-c0e47e9c86c7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:11.928348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c1193-7139-49a1-94e5-c822dc07d0fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:11.936457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-7e8c1193-7139-49a1-94e5-c822dc07d0fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:11.936457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:11.936457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:11.936457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:11.936457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:11.936457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:11.936457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:11.936457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:11.936457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:11.936457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:11.936457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:11.936457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:11.936457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:11.936457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:11.936457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:11.936457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:11.937617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:11.937617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:11.937617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:11.937617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:11.937617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:11.937617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:11.937617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:11.937617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:11.937617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:11.937617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:11.937617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:11.937617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:11.937617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:11.937617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:11.937617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:11.937617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:11.939132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:11.939132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:11.939132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:11.939132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:11.939132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:11.939132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:11.939132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:11.939132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:11.939132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:11.939132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:11.939132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:11.939132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:11.940869 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 57/118] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:28:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:11.941790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c1193-7139-49a1-94e5-c822dc07d0fa None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:11.965746 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bfa53-0562-4e2e-b87f-0b1a35ade678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:11.967497 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bfa53-0562-4e2e-b87f-0b1a35ade678 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.968103 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bfa53-0562-4e2e-b87f-0b1a35ade678 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.968637 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bfa53-0562-4e2e-b87f-0b1a35ade678 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.984482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053bfa53-0562-4e2e-b87f-0b1a35ade678 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:11.989075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:11.998469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-053bfa53-0562-4e2e-b87f-0b1a35ade678 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:11.998469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:11.998469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:11.998469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:11.998469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:11.998469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:11.998469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:11.998469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:11.998469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:11.998469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:11.998469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:11.998469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:11.998469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:11.998469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:11.998469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:11.998469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:11.999577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:11.999577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:11.999577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:11.999577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:11.999577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:11.999577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:11.999577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:11.999577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:11.999577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:11.999577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:11.999577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:11.999577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:11.999577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:11.999577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:11.999577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:11.999577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:12.001109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:12.001109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:12.001109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:12.001109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:12.001109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:12.001109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:12.001109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:12.001109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:12.001109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:12.001109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:12.001109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:12.001109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:12.003851 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 62/119] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:28:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:12.005228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bfa53-0562-4e2e-b87f-0b1a35ade678 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:12.034465 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:12.072919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c1193-7139-49a1-94e5-c822dc07d0fa None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:12.122755 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89f20f-d36c-4f74-8f5b-c10910bfa6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:12.123637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89f20f-d36c-4f74-8f5b-c10910bfa6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:12.123921 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89f20f-d36c-4f74-8f5b-c10910bfa6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:12.124199 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89f20f-d36c-4f74-8f5b-c10910bfa6e7 None admin] PATH_INFO: `/v3/users/febf11689a9345e18b3809a7135a9b7f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:12.126097 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89f20f-d36c-4f74-8f5b-c10910bfa6e7 None admin] RBAC: Authorizing `identity:get_user(user_id=febf11689a9345e18b3809a7135a9b7f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:12.127569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89f20f-d36c-4f74-8f5b-c10910bfa6e7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:12.129827 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 58/120] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:11 2022] GET /identity/v3/users/febf11689a9345e18b3809a7135a9b7f => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:12.130646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89f20f-d36c-4f74-8f5b-c10910bfa6e7 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:12.147724 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bfa53-0562-4e2e-b87f-0b1a35ade678 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:12.183578 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e66c90-5210-451c-ab75-96fc8d820490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:12.184228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e66c90-5210-451c-ab75-96fc8d820490 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:12.184521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e66c90-5210-451c-ab75-96fc8d820490 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:12.184804 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e66c90-5210-451c-ab75-96fc8d820490 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:12.185568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e66c90-5210-451c-ab75-96fc8d820490 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:12.185878 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e66c90-5210-451c-ab75-96fc8d820490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:12.187318 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e66c90-5210-451c-ab75-96fc8d820490 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:12.196737 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 63/121] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:12.198049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20c851-1620-4bcf-af2b-52bb0a1e6806 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:12.198323 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20c851-1620-4bcf-af2b-52bb0a1e6806 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:12.198605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20c851-1620-4bcf-af2b-52bb0a1e6806 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:12.199906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 64/122] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:12.200603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20c851-1620-4bcf-af2b-52bb0a1e6806 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:12.238275 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89f20f-d36c-4f74-8f5b-c10910bfa6e7 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:12.266357 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:12.273718 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a641577-5d8d-4eae-96f8-6015f4dd29b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:12.274430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a641577-5d8d-4eae-96f8-6015f4dd29b4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:12.274727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a641577-5d8d-4eae-96f8-6015f4dd29b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:12.275014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a641577-5d8d-4eae-96f8-6015f4dd29b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:12.275772 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a641577-5d8d-4eae-96f8-6015f4dd29b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:12.276144 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a641577-5d8d-4eae-96f8-6015f4dd29b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:12.277831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a641577-5d8d-4eae-96f8-6015f4dd29b4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:12.287227 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 59/123] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:28:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:12.288442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1590749-6ba9-4ad9-b243-678385a44589 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:12.288748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1590749-6ba9-4ad9-b243-678385a44589 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:12.289074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:12.290194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1590749-6ba9-4ad9-b243-678385a44589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:12.308151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d1590749-6ba9-4ad9-b243-678385a44589 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:12.309617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d1590749-6ba9-4ad9-b243-678385a44589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:12.320063 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20c851-1620-4bcf-af2b-52bb0a1e6806 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:12.364924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d95d4-8648-4fe7-b7e3-4f9b2970c302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:12.366850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d95d4-8648-4fe7-b7e3-4f9b2970c302 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:12.367316 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d95d4-8648-4fe7-b7e3-4f9b2970c302 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:12.367774 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d95d4-8648-4fe7-b7e3-4f9b2970c302 None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:12.369744 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d95d4-8648-4fe7-b7e3-4f9b2970c302 None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:12.371435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d95d4-8648-4fe7-b7e3-4f9b2970c302 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:12.374193 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 65/124] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:12 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:12.376161 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d95d4-8648-4fe7-b7e3-4f9b2970c302 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:12.394535 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 60/125] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:12.398490 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1590749-6ba9-4ad9-b243-678385a44589 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:12.515514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d95d4-8648-4fe7-b7e3-4f9b2970c302 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:12.535307 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:12.539321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1590749-6ba9-4ad9-b243-678385a44589 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:12.561057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a64d3b-0ef6-4c60-bdcf-eaae250e9487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:12.562294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a64d3b-0ef6-4c60-bdcf-eaae250e9487 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:12.562832 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a64d3b-0ef6-4c60-bdcf-eaae250e9487 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:12.563287 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a64d3b-0ef6-4c60-bdcf-eaae250e9487 None admin] PATH_INFO: `/v3/users/0ed92bd32dc24cfabfcf0a70a4518dcf` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:12.564742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a64d3b-0ef6-4c60-bdcf-eaae250e9487 None admin] RBAC: Authorizing `identity:get_user(user_id=0ed92bd32dc24cfabfcf0a70a4518dcf)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:12.566496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a64d3b-0ef6-4c60-bdcf-eaae250e9487 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:12.568371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 66/126] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:12 2022] GET /identity/v3/users/0ed92bd32dc24cfabfcf0a70a4518dcf => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:12.573703 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a64d3b-0ef6-4c60-bdcf-eaae250e9487 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:12.589164 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf644c5-7da9-4591-bb08-fd833b358e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:12.590095 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf644c5-7da9-4591-bb08-fd833b358e29 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:12.590383 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf644c5-7da9-4591-bb08-fd833b358e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:12.590663 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf644c5-7da9-4591-bb08-fd833b358e29 None admin] PATH_INFO: `/v3/users/febf11689a9345e18b3809a7135a9b7f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:12.591545 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:12.592993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf644c5-7da9-4591-bb08-fd833b358e29 None admin] RBAC: Authorizing `identity:get_user(user_id=febf11689a9345e18b3809a7135a9b7f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:12.595963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf644c5-7da9-4591-bb08-fd833b358e29 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:12.598490 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 61/127] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:12 2022] GET /identity/v3/users/febf11689a9345e18b3809a7135a9b7f => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:12.599725 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf644c5-7da9-4591-bb08-fd833b358e29 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:12.693634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a64d3b-0ef6-4c60-bdcf-eaae250e9487 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:12.729793 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72138b83-8b8a-4acb-b582-a5cae09b7990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:12.731068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72138b83-8b8a-4acb-b582-a5cae09b7990 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:12.731620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72138b83-8b8a-4acb-b582-a5cae09b7990 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:12.732104 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72138b83-8b8a-4acb-b582-a5cae09b7990 None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3/users/febf11689a9345e18b3809a7135a9b7f/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:12.733671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf644c5-7da9-4591-bb08-fd833b358e29 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:12.735303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72138b83-8b8a-4acb-b582-a5cae09b7990 None admin] RBAC: Authorizing `identity:create_grant(project_id=abaed0828ac34adb801becefa86d65b3, user_id=febf11689a9345e18b3809a7135a9b7f, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:12.736990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72138b83-8b8a-4acb-b582-a5cae09b7990 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:12.754201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 67/128] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:28:12 2022] PUT /identity/v3/projects/abaed0828ac34adb801becefa86d65b3/users/febf11689a9345e18b3809a7135a9b7f/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:12.756316 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72138b83-8b8a-4acb-b582-a5cae09b7990 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:12.779885 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698f6b8-b7c5-4abb-ac4f-17cd9b663e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:12.780600 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698f6b8-b7c5-4abb-ac4f-17cd9b663e44 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:12.780903 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698f6b8-b7c5-4abb-ac4f-17cd9b663e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:12.781168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698f6b8-b7c5-4abb-ac4f-17cd9b663e44 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:12.804200 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698f6b8-b7c5-4abb-ac4f-17cd9b663e44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:12.819543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-c698f6b8-b7c5-4abb-ac4f-17cd9b663e44 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:12.819543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:12.819543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:12.819543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:12.819543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:12.819543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:12.819543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:12.819543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:12.819543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:12.819543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:12.819543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:12.819543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:12.819543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:12.819543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:12.819543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:12.819543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:12.820980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:12.820980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:12.820980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:12.820980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:12.820980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:12.820980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:12.820980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:12.820980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:12.820980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:12.820980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:12.820980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:12.820980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:12.820980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:12.820980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:12.820980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:12.820980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:12.823129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:12.823129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:12.823129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:12.823129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:12.823129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:12.823129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:12.823129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:12.823129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:12.823129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:12.823129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:12.823129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:12.823129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:12.823129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 62/129] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:12.830373 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698f6b8-b7c5-4abb-ac4f-17cd9b663e44 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:12.868967 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:12.889821 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:12.936479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72138b83-8b8a-4acb-b582-a5cae09b7990 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:12.999070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20791155-85b0-4875-b685-f12023304529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:12.999955 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20791155-85b0-4875-b685-f12023304529 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:13.000294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20791155-85b0-4875-b685-f12023304529 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:13.000568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20791155-85b0-4875-b685-f12023304529 None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:13.002644 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20791155-85b0-4875-b685-f12023304529 None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:13.004414 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20791155-85b0-4875-b685-f12023304529 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:13.004991 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698f6b8-b7c5-4abb-ac4f-17cd9b663e44 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:13.006361 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 68/130] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:12 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:13.010755 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20791155-85b0-4875-b685-f12023304529 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:13.055327 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2050d4-5a14-4177-bb3a-fdbe59a59fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:13.056239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2050d4-5a14-4177-bb3a-fdbe59a59fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:13.056536 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2050d4-5a14-4177-bb3a-fdbe59a59fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:13.056821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2050d4-5a14-4177-bb3a-fdbe59a59fa9 None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:13.058566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2050d4-5a14-4177-bb3a-fdbe59a59fa9 None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:13.060469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2050d4-5a14-4177-bb3a-fdbe59a59fa9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:13.062940 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 63/131] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:12 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:13.063713 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2050d4-5a14-4177-bb3a-fdbe59a59fa9 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:13.168115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20791155-85b0-4875-b685-f12023304529 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:13.225026 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385998ba-a8a6-4b9c-9fa2-c61b103ac2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:13.225712 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385998ba-a8a6-4b9c-9fa2-c61b103ac2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:13.226019 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385998ba-a8a6-4b9c-9fa2-c61b103ac2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:13.226303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385998ba-a8a6-4b9c-9fa2-c61b103ac2e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:13.227055 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385998ba-a8a6-4b9c-9fa2-c61b103ac2e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:13.231105 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385998ba-a8a6-4b9c-9fa2-c61b103ac2e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:13.233936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385998ba-a8a6-4b9c-9fa2-c61b103ac2e1 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:13.250650 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:13.253155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 69/132] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:13.254090 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385998ba-a8a6-4b9c-9fa2-c61b103ac2e1 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:13.261855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2050d4-5a14-4177-bb3a-fdbe59a59fa9 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:13.274336 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:13.327211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f591d-55a2-48f5-bace-d98364a3c22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:13.328035 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f591d-55a2-48f5-bace-d98364a3c22f None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:13.328888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f591d-55a2-48f5-bace-d98364a3c22f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:13.329172 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f591d-55a2-48f5-bace-d98364a3c22f None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3/users/0ed92bd32dc24cfabfcf0a70a4518dcf/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:13.335637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f591d-55a2-48f5-bace-d98364a3c22f None admin] RBAC: Authorizing `identity:create_grant(project_id=abaed0828ac34adb801becefa86d65b3, user_id=0ed92bd32dc24cfabfcf0a70a4518dcf, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:13.338723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f591d-55a2-48f5-bace-d98364a3c22f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:13.364816 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 64/133] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:28:13 2022] PUT /identity/v3/projects/abaed0828ac34adb801becefa86d65b3/users/0ed92bd32dc24cfabfcf0a70a4518dcf/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:13.365733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f591d-55a2-48f5-bace-d98364a3c22f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:13.462286 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385998ba-a8a6-4b9c-9fa2-c61b103ac2e1 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:13.530979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88e24d-4f38-4001-9654-ff755aa4cae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:13.531944 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88e24d-4f38-4001-9654-ff755aa4cae9 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:13.532232 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88e24d-4f38-4001-9654-ff755aa4cae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:13.533667 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88e24d-4f38-4001-9654-ff755aa4cae9 None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3/users/febf11689a9345e18b3809a7135a9b7f/roles/dbe30950eded4c9ab2bcc0f7545b9bae` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:13.536986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88e24d-4f38-4001-9654-ff755aa4cae9 None admin] RBAC: Authorizing `identity:create_grant(project_id=abaed0828ac34adb801becefa86d65b3, user_id=febf11689a9345e18b3809a7135a9b7f, role_id=dbe30950eded4c9ab2bcc0f7545b9bae)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:13.538542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88e24d-4f38-4001-9654-ff755aa4cae9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:13.564156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 70/134] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:28:13 2022] PUT /identity/v3/projects/abaed0828ac34adb801becefa86d65b3/users/febf11689a9345e18b3809a7135a9b7f/roles/dbe30950eded4c9ab2bcc0f7545b9bae => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:13.575080 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f591d-55a2-48f5-bace-d98364a3c22f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:13.609374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d5eb7-837d-41bc-972f-ee7c4f913542 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:13.610335 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d5eb7-837d-41bc-972f-ee7c4f913542 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:13.610622 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d5eb7-837d-41bc-972f-ee7c4f913542 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:13.612178 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 71/135] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:28:13.618853 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a80a1-e2ea-4ead-9543-e42c86539506 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:13.619164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a80a1-e2ea-4ead-9543-e42c86539506 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:13.619428 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a80a1-e2ea-4ead-9543-e42c86539506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:13.646235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-611a80a1-e2ea-4ead-9543-e42c86539506 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:13.647487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-611a80a1-e2ea-4ead-9543-e42c86539506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:13.653422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e2a16-cea5-4755-a83b-69de0a20e96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:13.654378 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e2a16-cea5-4755-a83b-69de0a20e96f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:13.654664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e2a16-cea5-4755-a83b-69de0a20e96f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:13.654943 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e2a16-cea5-4755-a83b-69de0a20e96f None admin] PATH_INFO: `/v3/users/febf11689a9345e18b3809a7135a9b7f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:13.656248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e2a16-cea5-4755-a83b-69de0a20e96f None admin] RBAC: Authorizing `identity:get_user(user_id=febf11689a9345e18b3809a7135a9b7f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:13.657806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e2a16-cea5-4755-a83b-69de0a20e96f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:13.660710 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 65/136] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:13 2022] GET /identity/v3/users/febf11689a9345e18b3809a7135a9b7f => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:13.666234 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e2a16-cea5-4755-a83b-69de0a20e96f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:13.705348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:13.748857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:13.770229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 72/137] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:13.803093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a80a1-e2ea-4ead-9543-e42c86539506 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:13.809515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e2a16-cea5-4755-a83b-69de0a20e96f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:13.874152 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f89d6-6b8b-4a6c-8607-d5e0aeb6c9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:13.874963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f89d6-6b8b-4a6c-8607-d5e0aeb6c9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:13.875817 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f89d6-6b8b-4a6c-8607-d5e0aeb6c9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:13.876098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f89d6-6b8b-4a6c-8607-d5e0aeb6c9d7 None admin] PATH_INFO: `/v3/projects/b9b42dc83e944315b0439a61f932334f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:13.878975 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f89d6-6b8b-4a6c-8607-d5e0aeb6c9d7 None admin] RBAC: Authorizing `identity:get_project(project_id=b9b42dc83e944315b0439a61f932334f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:13.880469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f89d6-6b8b-4a6c-8607-d5e0aeb6c9d7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:13.887336 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 66/138] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:13 2022] GET /identity/v3/projects/b9b42dc83e944315b0439a61f932334f => generated 312 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:13.894347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f89d6-6b8b-4a6c-8607-d5e0aeb6c9d7 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:13.996100 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a80a1-e2ea-4ead-9543-e42c86539506 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:14.008489 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:14.018244 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f89d6-6b8b-4a6c-8607-d5e0aeb6c9d7 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:14.056414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4cddb-4fea-49ec-8884-c5011bc56e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:14.057327 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4cddb-4fea-49ec-8884-c5011bc56e14 None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:14.057612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4cddb-4fea-49ec-8884-c5011bc56e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:14.057889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4cddb-4fea-49ec-8884-c5011bc56e14 None admin] PATH_INFO: `/v3/projects/b9b42dc83e944315b0439a61f932334f/users/febf11689a9345e18b3809a7135a9b7f/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:14.059964 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4cddb-4fea-49ec-8884-c5011bc56e14 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9b42dc83e944315b0439a61f932334f, user_id=febf11689a9345e18b3809a7135a9b7f, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:14.062322 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4cddb-4fea-49ec-8884-c5011bc56e14 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:14.079415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 67/139] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:28:13 2022] PUT /identity/v3/projects/b9b42dc83e944315b0439a61f932334f/users/febf11689a9345e18b3809a7135a9b7f/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:14.116338 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a222493-3266-4523-8318-26a900eca056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:14.117194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a222493-3266-4523-8318-26a900eca056 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:14.117481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a222493-3266-4523-8318-26a900eca056 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:14.117760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a222493-3266-4523-8318-26a900eca056 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:14.122915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a222493-3266-4523-8318-26a900eca056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:14.124490 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a222493-3266-4523-8318-26a900eca056 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:14.126521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 73/140] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:28:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:14.132685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4cddb-4fea-49ec-8884-c5011bc56e14 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:14.274213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4cddb-4fea-49ec-8884-c5011bc56e14 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:14.321996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286873c-fff9-4d62-936b-b4d4805b166e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:14.322697 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286873c-fff9-4d62-936b-b4d4805b166e None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:14.322987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286873c-fff9-4d62-936b-b4d4805b166e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:14.323266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286873c-fff9-4d62-936b-b4d4805b166e None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:14.324136 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286873c-fff9-4d62-936b-b4d4805b166e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:14.325720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286873c-fff9-4d62-936b-b4d4805b166e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:14.354777 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 68/141] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:28:14 2022] POST /identity/v3/users => generated 283 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:14.375514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8517c-20e7-4f90-a01b-af78dd194f2b None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:14.375803 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8517c-20e7-4f90-a01b-af78dd194f2b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:14.375996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8517c-20e7-4f90-a01b-af78dd194f2b None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:14.377582 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 74/142] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:14.383740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d144253-e364-44b4-bc3d-57ee5eade3ea None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:14.384039 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d144253-e364-44b4-bc3d-57ee5eade3ea None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:14.384304 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d144253-e364-44b4-bc3d-57ee5eade3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:14.409373 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7d144253-e364-44b4-bc3d-57ee5eade3ea None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:14.410714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7d144253-e364-44b4-bc3d-57ee5eade3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:14.413074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:14.486458 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 69/143] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:14.511453 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8517c-20e7-4f90-a01b-af78dd194f2b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:14.587108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:14.635652 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8517c-20e7-4f90-a01b-af78dd194f2b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:14.673388 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367ec7f-8a61-4bb2-83ee-19fb7c102690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:14.674337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367ec7f-8a61-4bb2-83ee-19fb7c102690 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:14.674762 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367ec7f-8a61-4bb2-83ee-19fb7c102690 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:14.675040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367ec7f-8a61-4bb2-83ee-19fb7c102690 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:14.686950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8367ec7f-8a61-4bb2-83ee-19fb7c102690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:14.695615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-8367ec7f-8a61-4bb2-83ee-19fb7c102690 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:14.695615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:14.695615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:14.695615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:14.695615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:14.695615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:14.695615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:14.695615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:14.695615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:14.695615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:14.695615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:14.695615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:14.695615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:14.695615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:14.695615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:14.695615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:14.696478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:14.696478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:14.696478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:14.696478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:14.696478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:14.696478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:14.696478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:14.696478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:14.696478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:14.696478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:14.696478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:14.696478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:14.696478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:14.696478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:14.696478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:14.696478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:14.697220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:14.697220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:14.697220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:14.697220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:14.697220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:14.697220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:14.697220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:14.697220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:14.697220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:14.697220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:14.697220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:14.697220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:14.697220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 75/144] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:14.708090 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d144253-e364-44b4-bc3d-57ee5eade3ea None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:14.811253 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d144253-e364-44b4-bc3d-57ee5eade3ea None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:14.848734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e2e91-b3f1-47c7-8537-3f175aeff370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:14.849475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e2e91-b3f1-47c7-8537-3f175aeff370 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:14.849761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e2e91-b3f1-47c7-8537-3f175aeff370 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:14.850068 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e2e91-b3f1-47c7-8537-3f175aeff370 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:14.850833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e2e91-b3f1-47c7-8537-3f175aeff370 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:14.851141 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e2e91-b3f1-47c7-8537-3f175aeff370 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:14.852601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e2e91-b3f1-47c7-8537-3f175aeff370 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:14.858898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:14.864998 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 70/145] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:14.868535 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367ec7f-8a61-4bb2-83ee-19fb7c102690 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:14.975524 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367ec7f-8a61-4bb2-83ee-19fb7c102690 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:14.996320 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:15.012302 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cd666-d0ef-486a-905f-39764857c89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:15.013250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cd666-d0ef-486a-905f-39764857c89e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:15.013537 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cd666-d0ef-486a-905f-39764857c89e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:15.013861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cd666-d0ef-486a-905f-39764857c89e None admin] PATH_INFO: `/v3/users/d9aec001af964278b6c0c22f50eddf75` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:15.015156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9cd666-d0ef-486a-905f-39764857c89e None admin] RBAC: Authorizing `identity:get_user(user_id=d9aec001af964278b6c0c22f50eddf75)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:15.017195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9cd666-d0ef-486a-905f-39764857c89e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:15.019121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 76/146] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:14 2022] GET /identity/v3/users/d9aec001af964278b6c0c22f50eddf75 => generated 289 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:15.024483 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e2e91-b3f1-47c7-8537-3f175aeff370 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:15.150176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e2e91-b3f1-47c7-8537-3f175aeff370 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:15.194291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5851e18-5059-46cd-93cd-f16ce551f930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:15.195174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5851e18-5059-46cd-93cd-f16ce551f930 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:15.195464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5851e18-5059-46cd-93cd-f16ce551f930 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:15.195744 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5851e18-5059-46cd-93cd-f16ce551f930 None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:15.196955 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5851e18-5059-46cd-93cd-f16ce551f930 None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:15.198532 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5851e18-5059-46cd-93cd-f16ce551f930 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:15.200346 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 71/147] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:15 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:15.206062 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cd666-d0ef-486a-905f-39764857c89e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:15.317289 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cd666-d0ef-486a-905f-39764857c89e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:15.352015 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594dd505-28fb-4815-8bf8-10c1cee595dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:15.352915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dd505-28fb-4815-8bf8-10c1cee595dc None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:15.353200 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dd505-28fb-4815-8bf8-10c1cee595dc None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:15.353476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dd505-28fb-4815-8bf8-10c1cee595dc None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3/users/d9aec001af964278b6c0c22f50eddf75/roles/3d88c071954947f99ede3e31fb58494a` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:15.355555 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594dd505-28fb-4815-8bf8-10c1cee595dc None admin] RBAC: Authorizing `identity:create_grant(project_id=abaed0828ac34adb801becefa86d65b3, user_id=d9aec001af964278b6c0c22f50eddf75, role_id=3d88c071954947f99ede3e31fb58494a)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:15.357016 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594dd505-28fb-4815-8bf8-10c1cee595dc None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:15.382154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 77/148] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:28:15 2022] PUT /identity/v3/projects/abaed0828ac34adb801becefa86d65b3/users/d9aec001af964278b6c0c22f50eddf75/roles/3d88c071954947f99ede3e31fb58494a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:15.548197 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10261a-58a1-4614-af86-4b9f2435ff82 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:15.548587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10261a-58a1-4614-af86-4b9f2435ff82 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:15.548769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10261a-58a1-4614-af86-4b9f2435ff82 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:15.550241 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 72/149] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:15.556805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b0edc-3aed-4d24-bdd7-f19f9b01592a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:15.557394 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b0edc-3aed-4d24-bdd7-f19f9b01592a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:15.557902 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b0edc-3aed-4d24-bdd7-f19f9b01592a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:15.583523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4e5b0edc-3aed-4d24-bdd7-f19f9b01592a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:15.585043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4e5b0edc-3aed-4d24-bdd7-f19f9b01592a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:15.645624 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:15.673602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 78/150] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:15.708818 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10261a-58a1-4614-af86-4b9f2435ff82 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:15.802270 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:15.843205 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10261a-58a1-4614-af86-4b9f2435ff82 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:15.891193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de612f47-77b8-44b6-8b10-52e9387d1dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:15.892007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de612f47-77b8-44b6-8b10-52e9387d1dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:15.892288 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de612f47-77b8-44b6-8b10-52e9387d1dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:15.892562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de612f47-77b8-44b6-8b10-52e9387d1dd1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:15.909638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de612f47-77b8-44b6-8b10-52e9387d1dd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:15.917685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-de612f47-77b8-44b6-8b10-52e9387d1dd1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:15.917685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:15.917685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:15.917685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:15.917685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:15.917685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:15.917685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:15.917685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:15.917685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:15.917685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:15.917685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:15.917685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:15.917685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:15.917685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:15.917685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:15.917685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:15.918589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:15.918589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:15.918589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:15.918589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:15.918589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:15.918589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:15.918589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:15.918589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:15.918589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:15.918589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:15.918589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:15.918589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:15.918589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:15.918589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:15.918589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:15.918589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:15.919427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:15.919427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:15.919427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:15.919427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:15.919427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:15.919427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:15.919427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:15.919427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:15.919427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:15.919427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:15.919427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:15.919427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:15.919427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 73/151] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:15.926471 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b0edc-3aed-4d24-bdd7-f19f9b01592a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:16.056018 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b0edc-3aed-4d24-bdd7-f19f9b01592a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:16.099395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31024c46-3c8c-47a3-9315-fa2f9790f1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:16.100414 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31024c46-3c8c-47a3-9315-fa2f9790f1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:16.100921 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31024c46-3c8c-47a3-9315-fa2f9790f1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:16.101413 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31024c46-3c8c-47a3-9315-fa2f9790f1d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:16.102424 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31024c46-3c8c-47a3-9315-fa2f9790f1d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:16.102935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31024c46-3c8c-47a3-9315-fa2f9790f1d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:16.104541 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31024c46-3c8c-47a3-9315-fa2f9790f1d8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:16.114089 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 79/152] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:16.120009 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de612f47-77b8-44b6-8b10-52e9387d1dd1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:16.232613 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de612f47-77b8-44b6-8b10-52e9387d1dd1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:16.262138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:16.272552 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed958e-0314-4dcd-80cd-288107cc4e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:16.273823 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed958e-0314-4dcd-80cd-288107cc4e1b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:16.274348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed958e-0314-4dcd-80cd-288107cc4e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:16.274825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed958e-0314-4dcd-80cd-288107cc4e1b None admin] PATH_INFO: `/v3/users/febf11689a9345e18b3809a7135a9b7f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:16.276275 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed958e-0314-4dcd-80cd-288107cc4e1b None admin] RBAC: Authorizing `identity:get_user(user_id=febf11689a9345e18b3809a7135a9b7f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:16.277978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed958e-0314-4dcd-80cd-288107cc4e1b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:16.280174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 74/153] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:16 2022] GET /identity/v3/users/febf11689a9345e18b3809a7135a9b7f => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:16.285975 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31024c46-3c8c-47a3-9315-fa2f9790f1d8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:16.332602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:16.399928 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31024c46-3c8c-47a3-9315-fa2f9790f1d8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:16.436032 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d8c8b-a18a-4e32-92a9-ac8a5027c4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:16.436696 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d8c8b-a18a-4e32-92a9-ac8a5027c4cf None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:16.436986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d8c8b-a18a-4e32-92a9-ac8a5027c4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:16.437264 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d8c8b-a18a-4e32-92a9-ac8a5027c4cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:16.533493 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d8c8b-a18a-4e32-92a9-ac8a5027c4cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:16.535350 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d8c8b-a18a-4e32-92a9-ac8a5027c4cf None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:16.594253 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:16.691851 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 80/154] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:16.697536 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed958e-0314-4dcd-80cd-288107cc4e1b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:16.826687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed958e-0314-4dcd-80cd-288107cc4e1b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:16.884130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf17995-5f41-4dba-bae4-3b13395dac9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:16.885004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf17995-5f41-4dba-bae4-3b13395dac9a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:16.885288 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf17995-5f41-4dba-bae4-3b13395dac9a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:16.885566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf17995-5f41-4dba-bae4-3b13395dac9a None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:16.890722 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf17995-5f41-4dba-bae4-3b13395dac9a None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:16.892200 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf17995-5f41-4dba-bae4-3b13395dac9a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:16.894251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 75/155] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:16 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:16.900745 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d8c8b-a18a-4e32-92a9-ac8a5027c4cf None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:16.980065 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5776ac-5a76-4ecb-bb8f-f11e8666419a None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:16.980674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5776ac-5a76-4ecb-bb8f-f11e8666419a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:16.981151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5776ac-5a76-4ecb-bb8f-f11e8666419a None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:16.983110 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 76/156] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:16.990643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7a333-50d9-4ba5-85fb-78c65ac99226 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:16.991249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7a333-50d9-4ba5-85fb-78c65ac99226 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:16.991729 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7a333-50d9-4ba5-85fb-78c65ac99226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:17.026205 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d2f7a333-50d9-4ba5-85fb-78c65ac99226 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:17.027748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d2f7a333-50d9-4ba5-85fb-78c65ac99226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:17.057214 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d8c8b-a18a-4e32-92a9-ac8a5027c4cf None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:17.134125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c7565-074c-426e-a77b-42c2194dc6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:17.135048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c7565-074c-426e-a77b-42c2194dc6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:17.135487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c7565-074c-426e-a77b-42c2194dc6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:17.136154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c7565-074c-426e-a77b-42c2194dc6e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:17.137130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c7565-074c-426e-a77b-42c2194dc6e3 None admin] RBAC: Adding query filter params (role.id=ed842da869724411a5b355cd12f1c5c8, scope.project.id=abaed0828ac34adb801becefa86d65b3, user.id=febf11689a9345e18b3809a7135a9b7f) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:17.137578 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c7565-074c-426e-a77b-42c2194dc6e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:17.138363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:17.139698 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c7565-074c-426e-a77b-42c2194dc6e3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:17.148328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 77/157] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:17.156835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 81/158] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:16 2022] GET /identity/v3/role_assignments?user.id=febf11689a9345e18b3809a7135a9b7f&scope.project.id=abaed0828ac34adb801becefa86d65b3&role.id=ed842da869724411a5b355cd12f1c5c8 => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:17.207596 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7a333-50d9-4ba5-85fb-78c65ac99226 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:17.331354 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7a333-50d9-4ba5-85fb-78c65ac99226 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:17.368813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877104d-c384-499f-80e0-ca789d56951c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:17.369620 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877104d-c384-499f-80e0-ca789d56951c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:17.369909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877104d-c384-499f-80e0-ca789d56951c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:17.370225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877104d-c384-499f-80e0-ca789d56951c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:17.382474 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877104d-c384-499f-80e0-ca789d56951c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:17.390961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-5877104d-c384-499f-80e0-ca789d56951c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:17.390961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:17.390961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:17.390961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:17.390961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:17.390961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:17.390961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:17.390961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:17.390961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:17.390961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:17.390961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:17.390961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:17.390961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:17.390961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:17.390961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:17.390961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:17.391822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:17.391822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:17.391822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:17.391822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:17.391822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:17.391822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:17.391822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:17.391822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:17.391822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:17.391822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:17.391822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:17.391822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:17.391822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:17.391822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:17.391822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:17.391822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:17.392625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:17.392625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:17.392625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:17.392625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:17.392625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:17.392625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:17.392625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:17.392625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:17.392625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:17.392625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:17.392625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:17.392625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:17.392625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 78/159] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:28:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:17.398112 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c7565-074c-426e-a77b-42c2194dc6e3 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:17.402009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:17.540788 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c7565-074c-426e-a77b-42c2194dc6e3 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:17.580512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57495027-6a33-430c-9891-a57256cda13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:17.581205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57495027-6a33-430c-9891-a57256cda13c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:17.581550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57495027-6a33-430c-9891-a57256cda13c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:17.581771 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57495027-6a33-430c-9891-a57256cda13c None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:17.582569 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57495027-6a33-430c-9891-a57256cda13c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:17.582881 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57495027-6a33-430c-9891-a57256cda13c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:17.584310 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57495027-6a33-430c-9891-a57256cda13c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:17.597408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 82/160] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:17.602691 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877104d-c384-499f-80e0-ca789d56951c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:17.671009 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:17.679024 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d672bca-ff88-407c-92eb-27d5bbf549ba None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:17.679365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d672bca-ff88-407c-92eb-27d5bbf549ba None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:17.679589 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d672bca-ff88-407c-92eb-27d5bbf549ba None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:17.681307 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 83/161] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:17.687834 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c99f4-875d-4660-9565-cf3c925b4c67 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:17.688131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c99f4-875d-4660-9565-cf3c925b4c67 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:17.688418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c99f4-875d-4660-9565-cf3c925b4c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:17.723765 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-812c99f4-875d-4660-9565-cf3c925b4c67 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:17.724988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-812c99f4-875d-4660-9565-cf3c925b4c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:17.767011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877104d-c384-499f-80e0-ca789d56951c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:17.848609 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:17.857014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5bdaf-963b-4ca1-8357-2ca21a727840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:17.857910 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5bdaf-963b-4ca1-8357-2ca21a727840 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:17.858218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5bdaf-963b-4ca1-8357-2ca21a727840 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:17.858495 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5bdaf-963b-4ca1-8357-2ca21a727840 None admin] PATH_INFO: `/v3/users/0ed92bd32dc24cfabfcf0a70a4518dcf` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:17.860590 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5bdaf-963b-4ca1-8357-2ca21a727840 None admin] RBAC: Authorizing `identity:get_user(user_id=0ed92bd32dc24cfabfcf0a70a4518dcf)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:17.867023 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5bdaf-963b-4ca1-8357-2ca21a727840 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:17.871011 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 84/162] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:17.872755 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 79/163] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:17 2022] GET /identity/v3/users/0ed92bd32dc24cfabfcf0a70a4518dcf => generated 247 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:17.878799 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c99f4-875d-4660-9565-cf3c925b4c67 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:17.920627 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5bdaf-963b-4ca1-8357-2ca21a727840 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:18.030210 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c99f4-875d-4660-9565-cf3c925b4c67 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:18.113095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b861b8-c6c6-4c28-ac6f-3f1dc9ea7b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:18.115139 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b861b8-c6c6-4c28-ac6f-3f1dc9ea7b69 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:18.115772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b861b8-c6c6-4c28-ac6f-3f1dc9ea7b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:18.116852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b861b8-c6c6-4c28-ac6f-3f1dc9ea7b69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:18.133256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5bdaf-963b-4ca1-8357-2ca21a727840 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:18.176621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:18.218416 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:18.224838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b861b8-c6c6-4c28-ac6f-3f1dc9ea7b69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:18.226655 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0585d0d-7240-465f-861e-f227bdafd759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:18.227518 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0585d0d-7240-465f-861e-f227bdafd759 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:18.227789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0585d0d-7240-465f-861e-f227bdafd759 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:18.228233 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0585d0d-7240-465f-861e-f227bdafd759 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:18.231369 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b861b8-c6c6-4c28-ac6f-3f1dc9ea7b69 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:18.270553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0585d0d-7240-465f-861e-f227bdafd759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:18.279171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-c0585d0d-7240-465f-861e-f227bdafd759 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:18.279171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:18.279171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:18.279171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:18.279171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:18.279171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:18.279171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:18.279171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:18.279171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:18.279171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:18.279171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:18.279171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:18.279171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:18.279171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:18.279171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:18.279171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:18.280255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:18.280255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:18.280255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:18.280255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:18.280255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:18.280255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:18.280255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:18.280255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:18.280255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:18.280255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:18.280255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:18.280255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:18.280255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:18.280255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:18.280255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:18.280255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:18.281256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:18.281256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:18.281256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:18.281256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:18.281256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:18.281256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:18.281256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:18.281256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:18.281256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:18.281256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:18.281256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:18.281256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:18.283542 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 80/164] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:28:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:28:18.289668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0585d0d-7240-465f-861e-f227bdafd759 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:18.372172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 85/165] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:18.378233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b861b8-c6c6-4c28-ac6f-3f1dc9ea7b69 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:18.516063 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:18.529476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b861b8-c6c6-4c28-ac6f-3f1dc9ea7b69 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:18.573511 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0585d0d-7240-465f-861e-f227bdafd759 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:18.600276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc1619-50df-4719-a64e-e611dfd694a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:18.601512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc1619-50df-4719-a64e-e611dfd694a6 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:18.602029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc1619-50df-4719-a64e-e611dfd694a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:18.602501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc1619-50df-4719-a64e-e611dfd694a6 None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:18.603889 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc1619-50df-4719-a64e-e611dfd694a6 None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:18.605719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc1619-50df-4719-a64e-e611dfd694a6 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:18.607696 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 86/166] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:18 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:18.615260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc1619-50df-4719-a64e-e611dfd694a6 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:18.632254 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f9da0-548c-43fb-a9bd-525ecafc099f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:18.632851 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f9da0-548c-43fb-a9bd-525ecafc099f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:18.633133 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f9da0-548c-43fb-a9bd-525ecafc099f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:18.633404 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f9da0-548c-43fb-a9bd-525ecafc099f None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:18.634364 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f9da0-548c-43fb-a9bd-525ecafc099f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:18.634817 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f9da0-548c-43fb-a9bd-525ecafc099f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:18.636346 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f9da0-548c-43fb-a9bd-525ecafc099f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:18.649497 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 81/167] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:28:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:18.655913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f9da0-548c-43fb-a9bd-525ecafc099f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:18.735094 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:18.805990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc1619-50df-4719-a64e-e611dfd694a6 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:18.840100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f9da0-548c-43fb-a9bd-525ecafc099f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:18.876989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d70d3-4ca2-4031-9057-07dfa21d27b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:18.877655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d70d3-4ca2-4031-9057-07dfa21d27b2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:18.877940 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d70d3-4ca2-4031-9057-07dfa21d27b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:18.878237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d70d3-4ca2-4031-9057-07dfa21d27b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:18.879330 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d70d3-4ca2-4031-9057-07dfa21d27b2 None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=abaed0828ac34adb801becefa86d65b3, user.id=0ed92bd32dc24cfabfcf0a70a4518dcf) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:18.879642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d70d3-4ca2-4031-9057-07dfa21d27b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:18.881120 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d70d3-4ca2-4031-9057-07dfa21d27b2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:18.892031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 87/168] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:18 2022] GET /identity/v3/role_assignments?user.id=0ed92bd32dc24cfabfcf0a70a4518dcf&scope.project.id=abaed0828ac34adb801becefa86d65b3&role.id=864a6966630544e8bd68bcafba905c6f => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:18.893272 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665fa62-138b-4019-b9f5-f4369c0a5c96 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:18.893567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665fa62-138b-4019-b9f5-f4369c0a5c96 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:18.893846 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665fa62-138b-4019-b9f5-f4369c0a5c96 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:18.895308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 88/169] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:18.905854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d1d76-dd3c-4bef-82fe-bdf2220a8870 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:18.906750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d1d76-dd3c-4bef-82fe-bdf2220a8870 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:18.907027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d1d76-dd3c-4bef-82fe-bdf2220a8870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:18.911617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d80efd-e479-4178-b217-72d5bd7b5981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:18.912516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d80efd-e479-4178-b217-72d5bd7b5981 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:18.912796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d80efd-e479-4178-b217-72d5bd7b5981 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:18.913071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d80efd-e479-4178-b217-72d5bd7b5981 None admin] PATH_INFO: `/v3/users/febf11689a9345e18b3809a7135a9b7f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:18.914581 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d80efd-e479-4178-b217-72d5bd7b5981 None admin] RBAC: Authorizing `identity:get_user(user_id=febf11689a9345e18b3809a7135a9b7f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:18.916091 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d80efd-e479-4178-b217-72d5bd7b5981 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:18.917949 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 82/170] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:18 2022] GET /identity/v3/users/febf11689a9345e18b3809a7135a9b7f => generated 275 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:18.926465 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d80efd-e479-4178-b217-72d5bd7b5981 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:18.955162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1f7d1d76-dd3c-4bef-82fe-bdf2220a8870 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:18.956659 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1f7d1d76-dd3c-4bef-82fe-bdf2220a8870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:19.009142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:19.120723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 89/171] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:19.158330 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d1d76-dd3c-4bef-82fe-bdf2220a8870 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:19.163490 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d80efd-e479-4178-b217-72d5bd7b5981 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:19.183621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:19.229499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c0a3f-ebb0-4986-85df-d8bbe9f21ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:19.230201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c0a3f-ebb0-4986-85df-d8bbe9f21ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:19.230491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c0a3f-ebb0-4986-85df-d8bbe9f21ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:19.230758 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c0a3f-ebb0-4986-85df-d8bbe9f21ce4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:19.355071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c0a3f-ebb0-4986-85df-d8bbe9f21ce4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:19.356812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c0a3f-ebb0-4986-85df-d8bbe9f21ce4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:19.410814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d1d76-dd3c-4bef-82fe-bdf2220a8870 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:19.450375 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09006e0-b734-4fc7-aa5b-d1ac4ebdc040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:19.451795 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09006e0-b734-4fc7-aa5b-d1ac4ebdc040 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:19.452195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09006e0-b734-4fc7-aa5b-d1ac4ebdc040 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:19.452842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09006e0-b734-4fc7-aa5b-d1ac4ebdc040 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:19.467842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09006e0-b734-4fc7-aa5b-d1ac4ebdc040 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:19.478806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-c09006e0-b734-4fc7-aa5b-d1ac4ebdc040 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:19.478806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:19.478806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:19.478806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:19.478806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:19.478806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:19.478806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:19.478806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:19.478806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:19.478806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:19.478806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:19.478806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:19.478806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:19.478806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:19.478806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:19.478806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:19.479728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:19.479728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:19.479728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:19.479728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:19.479728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:19.479728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:19.479728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:19.479728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:19.479728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:19.479728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:19.479728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:19.479728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:19.479728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:19.479728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:19.479728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:19.479728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:19.480592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:19.480592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:19.480592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:19.480592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:19.480592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:19.480592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:19.480592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:19.480592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:19.480592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:19.480592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:19.480592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:19.480592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:19.480592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 90/172] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:19.481933 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a3a7c-192c-4890-9426-d307c312af6e None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:19.482260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a3a7c-192c-4890-9426-d307c312af6e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:19.482539 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a3a7c-192c-4890-9426-d307c312af6e None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:19.484292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 91/173] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:19.486268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c77905-9ae0-4da6-84bf-7dc54e26efcf None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:19.486559 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c77905-9ae0-4da6-84bf-7dc54e26efcf None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:19.487011 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c77905-9ae0-4da6-84bf-7dc54e26efcf None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:19.488589 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 92/174] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:19.494434 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2ab69-6ee3-4353-833c-33503dc8adfe None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:19.494733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2ab69-6ee3-4353-833c-33503dc8adfe None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:19.495068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2ab69-6ee3-4353-833c-33503dc8adfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:19.513313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 83/175] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:19.514535 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c0a3f-ebb0-4986-85df-d8bbe9f21ce4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:19.517253 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b0e2ab69-6ee3-4353-833c-33503dc8adfe None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:19.519713 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b0e2ab69-6ee3-4353-833c-33503dc8adfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:19.532307 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:19.536232 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:19.605982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 93/176] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:19.605982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db498a-9651-411e-9fe3-a563ab9bcb57 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:19.605982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db498a-9651-411e-9fe3-a563ab9bcb57 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:19.605982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db498a-9651-411e-9fe3-a563ab9bcb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:19.624019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c0a3f-ebb0-4986-85df-d8bbe9f21ce4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:19.627933 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-93db498a-9651-411e-9fe3-a563ab9bcb57 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:19.629393 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-93db498a-9651-411e-9fe3-a563ab9bcb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:19.663642 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1042e2f-f6bf-4c1a-9009-696a3d648a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:19.664237 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1042e2f-f6bf-4c1a-9009-696a3d648a2d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:19.664552 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1042e2f-f6bf-4c1a-9009-696a3d648a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:19.664796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1042e2f-f6bf-4c1a-9009-696a3d648a2d None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:19.665544 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1042e2f-f6bf-4c1a-9009-696a3d648a2d None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:19.665852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1042e2f-f6bf-4c1a-9009-696a3d648a2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:19.667296 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1042e2f-f6bf-4c1a-9009-696a3d648a2d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:19.678629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 84/177] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:19.679769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf384a8-d40c-4e63-8d40-041725ba4d64 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:19.680066 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf384a8-d40c-4e63-8d40-041725ba4d64 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:19.680348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf384a8-d40c-4e63-8d40-041725ba4d64 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:19.681933 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 85/178] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:19.682554 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf384a8-d40c-4e63-8d40-041725ba4d64 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:19.715004 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 94/179] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:19.722863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db498a-9651-411e-9fe3-a563ab9bcb57 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:19.770487 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:19.799115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf384a8-d40c-4e63-8d40-041725ba4d64 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:19.820289 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:19.837101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6fa05-0143-4588-942a-df2f512fe46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:19.837991 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6fa05-0143-4588-942a-df2f512fe46e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:19.838275 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6fa05-0143-4588-942a-df2f512fe46e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:19.838694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6fa05-0143-4588-942a-df2f512fe46e None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:19.839745 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6fa05-0143-4588-942a-df2f512fe46e None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:19.842869 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6fa05-0143-4588-942a-df2f512fe46e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:19.843010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 86/180] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:19 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:19.847043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db498a-9651-411e-9fe3-a563ab9bcb57 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:19.848503 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6fa05-0143-4588-942a-df2f512fe46e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:19.884145 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758f8a6-63cf-4337-95b8-ff37747cceaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:19.885069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758f8a6-63cf-4337-95b8-ff37747cceaa None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:19.885425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758f8a6-63cf-4337-95b8-ff37747cceaa None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:19.885766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758f8a6-63cf-4337-95b8-ff37747cceaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:19.896788 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758f8a6-63cf-4337-95b8-ff37747cceaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:19.904827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-6758f8a6-63cf-4337-95b8-ff37747cceaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:19.904827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:19.904827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:19.904827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:19.904827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:19.904827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:19.904827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:19.904827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:19.904827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:19.904827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:19.904827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:19.904827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:19.904827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:19.904827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:19.904827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:19.904827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:19.905824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:19.905824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:19.905824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:19.905824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:19.905824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:19.905824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:19.905824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:19.905824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:19.905824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:19.905824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:19.905824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:19.905824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:19.905824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:19.905824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:19.905824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:19.905824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:19.907206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:19.907206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:19.907206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:19.907206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:19.907206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:19.907206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:19.907206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:19.907206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:19.907206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:19.907206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:19.907206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:19.907206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:19.907206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 95/181] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:28:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:19.907895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39e347-d41a-4dd7-9a72-50a8d69f8edb None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:19.908276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39e347-d41a-4dd7-9a72-50a8d69f8edb None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:19.909052 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39e347-d41a-4dd7-9a72-50a8d69f8edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:19.926188 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7f39e347-d41a-4dd7-9a72-50a8d69f8edb None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:19.927290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7f39e347-d41a-4dd7-9a72-50a8d69f8edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:19.967504 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6fa05-0143-4588-942a-df2f512fe46e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:20.006990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 96/182] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:20.008753 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39e347-d41a-4dd7-9a72-50a8d69f8edb None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:20.024360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9238553-e65b-4f04-bba0-6f7653bed6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:20.025278 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9238553-e65b-4f04-bba0-6f7653bed6dc None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.025538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9238553-e65b-4f04-bba0-6f7653bed6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.025813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9238553-e65b-4f04-bba0-6f7653bed6dc None admin] PATH_INFO: `/v3/users/febf11689a9345e18b3809a7135a9b7f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.027091 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9238553-e65b-4f04-bba0-6f7653bed6dc None admin] RBAC: Authorizing `identity:get_user(user_id=febf11689a9345e18b3809a7135a9b7f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:20.028771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9238553-e65b-4f04-bba0-6f7653bed6dc None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:20.030470 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 87/183] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:19 2022] GET /identity/v3/users/febf11689a9345e18b3809a7135a9b7f => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:20.035434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9238553-e65b-4f04-bba0-6f7653bed6dc None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:20.045292 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:20.085008 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:20.119334 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39e347-d41a-4dd7-9a72-50a8d69f8edb None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:20.134265 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9238553-e65b-4f04-bba0-6f7653bed6dc None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:20.152492 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27abe1-ddbf-4cf7-b381-a52640227e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:20.153292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27abe1-ddbf-4cf7-b381-a52640227e69 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.153572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27abe1-ddbf-4cf7-b381-a52640227e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.153843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27abe1-ddbf-4cf7-b381-a52640227e69 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.165570 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27abe1-ddbf-4cf7-b381-a52640227e69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:20.167356 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca4db9-7af6-4f11-8508-56c10c1bde51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:20.168011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca4db9-7af6-4f11-8508-56c10c1bde51 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.168299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca4db9-7af6-4f11-8508-56c10c1bde51 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.168578 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca4db9-7af6-4f11-8508-56c10c1bde51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.169413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca4db9-7af6-4f11-8508-56c10c1bde51 None admin] RBAC: Adding query filter params (role.id=dbe30950eded4c9ab2bcc0f7545b9bae, scope.project.id=abaed0828ac34adb801becefa86d65b3, user.id=febf11689a9345e18b3809a7135a9b7f) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:20.169730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca4db9-7af6-4f11-8508-56c10c1bde51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:20.171238 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca4db9-7af6-4f11-8508-56c10c1bde51 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:20.174023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-aa27abe1-ddbf-4cf7-b381-a52640227e69 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:20.174023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:20.174023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:20.174023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:20.174023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:20.174023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:20.174023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:20.174023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:20.174023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:20.174023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:20.174023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:20.174023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:20.174023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:20.174023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:20.174023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:20.174023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:20.174891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:20.174891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:20.174891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:20.174891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:20.174891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:20.174891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:20.174891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:20.174891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:20.174891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:20.174891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:20.174891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:20.174891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:20.174891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:20.174891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:20.174891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:20.174891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:20.175668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:20.175668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:20.175668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:20.175668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:20.175668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:20.175668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:20.175668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:20.175668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:20.175668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:20.175668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:20.175668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:20.175668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:20.175668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 97/184] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:20.176381 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27abe1-ddbf-4cf7-b381-a52640227e69 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:20.190024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 88/185] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:20 2022] GET /identity/v3/role_assignments?user.id=febf11689a9345e18b3809a7135a9b7f&scope.project.id=abaed0828ac34adb801becefa86d65b3&role.id=dbe30950eded4c9ab2bcc0f7545b9bae => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:20.190024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca4db9-7af6-4f11-8508-56c10c1bde51 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:20.289761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:20.294058 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27abe1-ddbf-4cf7-b381-a52640227e69 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:20.320308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca4db9-7af6-4f11-8508-56c10c1bde51 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:20.327152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:20.338603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa21c26-2c73-4f1c-92da-1bc1dd913aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:20.339288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa21c26-2c73-4f1c-92da-1bc1dd913aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.339573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa21c26-2c73-4f1c-92da-1bc1dd913aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.339850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa21c26-2c73-4f1c-92da-1bc1dd913aa6 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.340610 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa21c26-2c73-4f1c-92da-1bc1dd913aa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:20.340928 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa21c26-2c73-4f1c-92da-1bc1dd913aa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:20.342368 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa21c26-2c73-4f1c-92da-1bc1dd913aa6 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:20.353216 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 98/186] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:20.353936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa21c26-2c73-4f1c-92da-1bc1dd913aa6 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:20.364565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232bc1ac-37a2-4f57-8ded-8a8f97f564ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:20.365390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bc1ac-37a2-4f57-8ded-8a8f97f564ca None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.365678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bc1ac-37a2-4f57-8ded-8a8f97f564ca None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.367238 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bc1ac-37a2-4f57-8ded-8a8f97f564ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.381302 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232bc1ac-37a2-4f57-8ded-8a8f97f564ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:20.389623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-232bc1ac-37a2-4f57-8ded-8a8f97f564ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:20.389623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:20.389623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:20.389623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:20.389623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:20.389623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:20.389623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:20.389623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:20.389623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:20.389623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:20.389623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:20.389623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:20.389623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:20.389623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:20.389623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:20.389623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:20.391191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:20.391191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:20.391191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:20.391191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:20.391191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:20.391191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:20.391191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:20.391191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:20.391191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:20.391191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:20.391191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:20.391191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:20.391191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:20.391191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:20.391191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:20.391191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:20.392048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:20.392048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:20.392048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:20.392048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:20.392048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:20.392048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:20.392048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:20.392048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:20.392048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:20.392048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:20.392048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:20.392048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:20.392048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 89/187] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:28:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:20.392829 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67de554-0ee3-484f-acb4-b0e68a2a286b None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.393125 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67de554-0ee3-484f-acb4-b0e68a2a286b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.393401 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67de554-0ee3-484f-acb4-b0e68a2a286b None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.394721 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 90/188] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:20.399563 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67de554-0ee3-484f-acb4-b0e68a2a286b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:20.467862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa21c26-2c73-4f1c-92da-1bc1dd913aa6 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:20.501574 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67de554-0ee3-484f-acb4-b0e68a2a286b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:20.502810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa45448-735c-494c-bd54-f6be5c67537e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:20.503721 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa45448-735c-494c-bd54-f6be5c67537e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.504692 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa45448-735c-494c-bd54-f6be5c67537e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.504692 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa45448-735c-494c-bd54-f6be5c67537e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.535689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b176a-3488-461e-8694-a66bbba0daf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:20.536347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b176a-3488-461e-8694-a66bbba0daf9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.536630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b176a-3488-461e-8694-a66bbba0daf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.536903 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b176a-3488-461e-8694-a66bbba0daf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.537645 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b176a-3488-461e-8694-a66bbba0daf9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:20.537973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b176a-3488-461e-8694-a66bbba0daf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:20.539370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b176a-3488-461e-8694-a66bbba0daf9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:20.540727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:20.548780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 91/189] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:20.549678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b176a-3488-461e-8694-a66bbba0daf9 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:20.563602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:20.574871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa45448-735c-494c-bd54-f6be5c67537e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:20.576393 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa45448-735c-494c-bd54-f6be5c67537e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:20.648910 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b176a-3488-461e-8694-a66bbba0daf9 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:20.675294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 99/190] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:20.676137 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa45448-735c-494c-bd54-f6be5c67537e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:20.688119 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07173cd6-ee73-4ef1-91fa-c1e0bbabefbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:20.688992 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07173cd6-ee73-4ef1-91fa-c1e0bbabefbb None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.689268 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07173cd6-ee73-4ef1-91fa-c1e0bbabefbb None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.689534 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07173cd6-ee73-4ef1-91fa-c1e0bbabefbb None admin] PATH_INFO: `/v3/users/b000c41b72b14a63999b9a2e3e47f989` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.702303 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07173cd6-ee73-4ef1-91fa-c1e0bbabefbb None admin] RBAC: Authorizing `identity:get_user(user_id=b000c41b72b14a63999b9a2e3e47f989)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:20.704356 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07173cd6-ee73-4ef1-91fa-c1e0bbabefbb None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:20.706072 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 92/191] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:20 2022] GET /identity/v3/users/b000c41b72b14a63999b9a2e3e47f989 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:20.707239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4fa2c-40e4-46ad-96c0-d1d15786ba94 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.707519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4fa2c-40e4-46ad-96c0-d1d15786ba94 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.707791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4fa2c-40e4-46ad-96c0-d1d15786ba94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.723690 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7ca4fa2c-40e4-46ad-96c0-d1d15786ba94 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:20.724829 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7ca4fa2c-40e4-46ad-96c0-d1d15786ba94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:20.769889 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:20.774524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:20.779575 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa45448-735c-494c-bd54-f6be5c67537e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:20.791957 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 93/192] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:20.792678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4fa2c-40e4-46ad-96c0-d1d15786ba94 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:20.822039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b776c0-730e-4b58-a334-fbc9d3e88a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:20.822794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b776c0-730e-4b58-a334-fbc9d3e88a0d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.823029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b776c0-730e-4b58-a334-fbc9d3e88a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.823378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b776c0-730e-4b58-a334-fbc9d3e88a0d None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.824121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b776c0-730e-4b58-a334-fbc9d3e88a0d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:20.824423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b776c0-730e-4b58-a334-fbc9d3e88a0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:20.825811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b776c0-730e-4b58-a334-fbc9d3e88a0d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:20.834717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 100/193] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:28:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:20.835510 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b776c0-730e-4b58-a334-fbc9d3e88a0d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:20.895178 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4fa2c-40e4-46ad-96c0-d1d15786ba94 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:20.927989 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eccf7a2-dbc4-4247-8561-9f212fbaea78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:20.928865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccf7a2-dbc4-4247-8561-9f212fbaea78 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.929170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccf7a2-dbc4-4247-8561-9f212fbaea78 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.929444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccf7a2-dbc4-4247-8561-9f212fbaea78 None admin] PATH_INFO: `/v3/users/d9aec001af964278b6c0c22f50eddf75` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.930659 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eccf7a2-dbc4-4247-8561-9f212fbaea78 None admin] RBAC: Authorizing `identity:get_user(user_id=d9aec001af964278b6c0c22f50eddf75)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:20.932098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eccf7a2-dbc4-4247-8561-9f212fbaea78 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:20.933723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 94/194] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:20 2022] GET /identity/v3/users/d9aec001af964278b6c0c22f50eddf75 => generated 289 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:20.934513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eccf7a2-dbc4-4247-8561-9f212fbaea78 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:20.941728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b776c0-730e-4b58-a334-fbc9d3e88a0d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:20.975436 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06ae3d-4450-4fc1-b2c8-5741423363f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:20.976233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06ae3d-4450-4fc1-b2c8-5741423363f7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.976516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06ae3d-4450-4fc1-b2c8-5741423363f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.976788 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06ae3d-4450-4fc1-b2c8-5741423363f7 None admin] PATH_INFO: `/v3/projects/b9b42dc83e944315b0439a61f932334f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.978001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06ae3d-4450-4fc1-b2c8-5741423363f7 None admin] RBAC: Authorizing `identity:get_project(project_id=b9b42dc83e944315b0439a61f932334f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:20.979478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06ae3d-4450-4fc1-b2c8-5741423363f7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:20.981179 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 101/195] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:20 2022] GET /identity/v3/projects/b9b42dc83e944315b0439a61f932334f => generated 312 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:20.983090 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06ae3d-4450-4fc1-b2c8-5741423363f7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:20.986686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:21.006164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:21.034369 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eccf7a2-dbc4-4247-8561-9f212fbaea78 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.067868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d7dbf-5b85-4721-8526-2e1ae4c99f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.068515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d7dbf-5b85-4721-8526-2e1ae4c99f72 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.068802 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d7dbf-5b85-4721-8526-2e1ae4c99f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.069079 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d7dbf-5b85-4721-8526-2e1ae4c99f72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.084295 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06ae3d-4450-4fc1-b2c8-5741423363f7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.116771 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a8171-b79c-454e-9e84-747b8a7d3e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.117579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a8171-b79c-454e-9e84-747b8a7d3e5b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.117861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a8171-b79c-454e-9e84-747b8a7d3e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.118171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a8171-b79c-454e-9e84-747b8a7d3e5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.119357 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a8171-b79c-454e-9e84-747b8a7d3e5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.120818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a8171-b79c-454e-9e84-747b8a7d3e5b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.122504 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 102/196] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:28:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:21.123256 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a8171-b79c-454e-9e84-747b8a7d3e5b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:21.142181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d7dbf-5b85-4721-8526-2e1ae4c99f72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.143669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d7dbf-5b85-4721-8526-2e1ae4c99f72 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.194850 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:21.214726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:21.229290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a8171-b79c-454e-9e84-747b8a7d3e5b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.247231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 95/197] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:21.248768 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d7dbf-5b85-4721-8526-2e1ae4c99f72 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:21.264184 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f01f9f-ef95-4dd6-a118-8dbe57b48f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.265069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f01f9f-ef95-4dd6-a118-8dbe57b48f4a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.265353 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f01f9f-ef95-4dd6-a118-8dbe57b48f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.265627 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f01f9f-ef95-4dd6-a118-8dbe57b48f4a None admin] PATH_INFO: `/v3/users/b000c41b72b14a63999b9a2e3e47f989` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.266850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f01f9f-ef95-4dd6-a118-8dbe57b48f4a None admin] RBAC: Authorizing `identity:get_user(user_id=b000c41b72b14a63999b9a2e3e47f989)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.268312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f01f9f-ef95-4dd6-a118-8dbe57b48f4a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.274080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 103/198] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:21 2022] GET /identity/v3/users/b000c41b72b14a63999b9a2e3e47f989 => generated 283 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:21.274080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f01f9f-ef95-4dd6-a118-8dbe57b48f4a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:21.353566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d7dbf-5b85-4721-8526-2e1ae4c99f72 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.379697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f01f9f-ef95-4dd6-a118-8dbe57b48f4a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.389512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038289b7-ded8-4e30-abb7-857095eebcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.390197 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038289b7-ded8-4e30-abb7-857095eebcbf None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.390983 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038289b7-ded8-4e30-abb7-857095eebcbf None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.391250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038289b7-ded8-4e30-abb7-857095eebcbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.415746 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274ed7b-db2b-4b7c-8041-4f0de1ffa671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.416404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274ed7b-db2b-4b7c-8041-4f0de1ffa671 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.416685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274ed7b-db2b-4b7c-8041-4f0de1ffa671 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.416959 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274ed7b-db2b-4b7c-8041-4f0de1ffa671 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.417816 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274ed7b-db2b-4b7c-8041-4f0de1ffa671 None admin] RBAC: Adding query filter params (role.id=ed842da869724411a5b355cd12f1c5c8, scope.project.id=b9b42dc83e944315b0439a61f932334f, user.id=febf11689a9345e18b3809a7135a9b7f) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:21.418159 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274ed7b-db2b-4b7c-8041-4f0de1ffa671 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.419624 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274ed7b-db2b-4b7c-8041-4f0de1ffa671 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.427454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:21.437942 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 104/199] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:21 2022] GET /identity/v3/role_assignments?user.id=febf11689a9345e18b3809a7135a9b7f&scope.project.id=b9b42dc83e944315b0439a61f932334f&role.id=ed842da869724411a5b355cd12f1c5c8 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:21.437942 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274ed7b-db2b-4b7c-8041-4f0de1ffa671 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:21.461678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:21.506390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038289b7-ded8-4e30-abb7-857095eebcbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.508178 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038289b7-ded8-4e30-abb7-857095eebcbf None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.558293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274ed7b-db2b-4b7c-8041-4f0de1ffa671 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.605399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cc8a7-9500-48e8-a8d6-adeaa5cc2969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.606145 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc8a7-9500-48e8-a8d6-adeaa5cc2969 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.606440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc8a7-9500-48e8-a8d6-adeaa5cc2969 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.606717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc8a7-9500-48e8-a8d6-adeaa5cc2969 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.607452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cc8a7-9500-48e8-a8d6-adeaa5cc2969 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.608980 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cc8a7-9500-48e8-a8d6-adeaa5cc2969 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.631863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 105/200] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:28:21 2022] POST /identity/v3/users => generated 297 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:21.632794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cc8a7-9500-48e8-a8d6-adeaa5cc2969 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:21.662948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 96/201] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:21.663672 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038289b7-ded8-4e30-abb7-857095eebcbf None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:21.738101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:21.747262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:21.776632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cc8a7-9500-48e8-a8d6-adeaa5cc2969 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.794533 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038289b7-ded8-4e30-abb7-857095eebcbf None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.815710 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2ee85-2353-42d6-bd0b-551350b84222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.816604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2ee85-2353-42d6-bd0b-551350b84222 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.816887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2ee85-2353-42d6-bd0b-551350b84222 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.817164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2ee85-2353-42d6-bd0b-551350b84222 None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.818483 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2ee85-2353-42d6-bd0b-551350b84222 None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.820377 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2ee85-2353-42d6-bd0b-551350b84222 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.821912 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 106/202] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:21 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:21.822610 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2ee85-2353-42d6-bd0b-551350b84222 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:21.837853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f60ed-212d-408f-abe3-bf414718739a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.838541 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f60ed-212d-408f-abe3-bf414718739a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.838821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f60ed-212d-408f-abe3-bf414718739a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.839648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f60ed-212d-408f-abe3-bf414718739a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.924869 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f60ed-212d-408f-abe3-bf414718739a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.926808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f60ed-212d-408f-abe3-bf414718739a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.952577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2ee85-2353-42d6-bd0b-551350b84222 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.988990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:21.989733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b96db4-01fb-4edd-8bb4-e6ba57992d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.991263 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b96db4-01fb-4edd-8bb4-e6ba57992d1f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.991662 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b96db4-01fb-4edd-8bb4-e6ba57992d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.992007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b96db4-01fb-4edd-8bb4-e6ba57992d1f None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.993397 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b96db4-01fb-4edd-8bb4-e6ba57992d1f None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.996232 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b96db4-01fb-4edd-8bb4-e6ba57992d1f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.996688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 107/203] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:21 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:21.998097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b96db4-01fb-4edd-8bb4-e6ba57992d1f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:22.021300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:22.063163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 97/204] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:22.064599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f60ed-212d-408f-abe3-bf414718739a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:22.132602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b96db4-01fb-4edd-8bb4-e6ba57992d1f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:22.176362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115638a-7ff4-48fc-9d1e-da432de5a3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:22.177492 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115638a-7ff4-48fc-9d1e-da432de5a3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:22.177783 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115638a-7ff4-48fc-9d1e-da432de5a3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:22.178110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115638a-7ff4-48fc-9d1e-da432de5a3f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:22.179288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115638a-7ff4-48fc-9d1e-da432de5a3f2 None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=ae719f7141f34603811594a41497e87c, user.id=b000c41b72b14a63999b9a2e3e47f989) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:22.179614 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115638a-7ff4-48fc-9d1e-da432de5a3f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:22.181321 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115638a-7ff4-48fc-9d1e-da432de5a3f2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:22.194002 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 108/205] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:21 2022] GET /identity/v3/role_assignments?user.id=b000c41b72b14a63999b9a2e3e47f989&scope.project.id=ae719f7141f34603811594a41497e87c&role.id=864a6966630544e8bd68bcafba905c6f => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:22.194002 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115638a-7ff4-48fc-9d1e-da432de5a3f2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:22.198951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f60ed-212d-408f-abe3-bf414718739a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:22.260616 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e78c06-f133-483e-9441-3f2810a4e428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:22.261289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e78c06-f133-483e-9441-3f2810a4e428 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:22.265987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e78c06-f133-483e-9441-3f2810a4e428 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:22.265987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e78c06-f133-483e-9441-3f2810a4e428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:22.265987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e78c06-f133-483e-9441-3f2810a4e428 None admin] RBAC: Adding query filter params (role.id=3d88c071954947f99ede3e31fb58494a, scope.project.id=abaed0828ac34adb801becefa86d65b3, user.id=d9aec001af964278b6c0c22f50eddf75) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:22.265987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e78c06-f133-483e-9441-3f2810a4e428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:22.265987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e78c06-f133-483e-9441-3f2810a4e428 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:22.279469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 98/206] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:22 2022] GET /identity/v3/role_assignments?user.id=d9aec001af964278b6c0c22f50eddf75&scope.project.id=abaed0828ac34adb801becefa86d65b3&role.id=3d88c071954947f99ede3e31fb58494a => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:22.320960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:22.348087 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115638a-7ff4-48fc-9d1e-da432de5a3f2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:22.390122 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbb585-343f-432e-bd5c-881cfe2bca0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:22.391512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbb585-343f-432e-bd5c-881cfe2bca0b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:22.391512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbb585-343f-432e-bd5c-881cfe2bca0b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:22.391676 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbb585-343f-432e-bd5c-881cfe2bca0b None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:22.393518 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbb585-343f-432e-bd5c-881cfe2bca0b None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:22.395064 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbb585-343f-432e-bd5c-881cfe2bca0b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:22.396927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 109/207] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:22 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:22.403017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e78c06-f133-483e-9441-3f2810a4e428 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:22.423807 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:22.512384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e78c06-f133-483e-9441-3f2810a4e428 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:22.548329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469aa95-974f-4ac1-a6d1-478c82a95f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:22.548988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469aa95-974f-4ac1-a6d1-478c82a95f9e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:22.549271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469aa95-974f-4ac1-a6d1-478c82a95f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:22.549541 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469aa95-974f-4ac1-a6d1-478c82a95f9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:22.550400 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469aa95-974f-4ac1-a6d1-478c82a95f9e None admin] RBAC: Adding query filter params (role.id=dbe30950eded4c9ab2bcc0f7545b9bae, scope.project.id=ae719f7141f34603811594a41497e87c, user.id=b000c41b72b14a63999b9a2e3e47f989) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:22.550709 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469aa95-974f-4ac1-a6d1-478c82a95f9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:22.552153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469aa95-974f-4ac1-a6d1-478c82a95f9e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:22.558018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 99/208] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:22 2022] GET /identity/v3/role_assignments?user.id=b000c41b72b14a63999b9a2e3e47f989&scope.project.id=ae719f7141f34603811594a41497e87c&role.id=dbe30950eded4c9ab2bcc0f7545b9bae => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:23.911830 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2b8d1-4c3d-476b-9f4e-6f1faaae3094 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:23.912514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2b8d1-4c3d-476b-9f4e-6f1faaae3094 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:23.912514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2b8d1-4c3d-476b-9f4e-6f1faaae3094 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:23.914088 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 110/209] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:23.919521 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecde9b4-7a01-467f-8d4a-31533d5aef37 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:23.920019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecde9b4-7a01-467f-8d4a-31533d5aef37 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:23.920170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecde9b4-7a01-467f-8d4a-31533d5aef37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:23.939961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-aecde9b4-7a01-467f-8d4a-31533d5aef37 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:23.941432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-aecde9b4-7a01-467f-8d4a-31533d5aef37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:23.990683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005e5e5-c056-49f5-95b8-a4e886c7f50d None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:23.991028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005e5e5-c056-49f5-95b8-a4e886c7f50d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:23.991251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005e5e5-c056-49f5-95b8-a4e886c7f50d None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:23.992693 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 111/210] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:23.997608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea714c-3e6e-4a89-9887-986e9e306f92 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:23.997897 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea714c-3e6e-4a89-9887-986e9e306f92 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:23.998217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea714c-3e6e-4a89-9887-986e9e306f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:24.013266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 100/211] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:24.019067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-bcea714c-3e6e-4a89-9887-986e9e306f92 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:24.020277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-bcea714c-3e6e-4a89-9887-986e9e306f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:24.037743 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecde9b4-7a01-467f-8d4a-31533d5aef37 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:24.050711 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:24.094223 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 112/212] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:24.118518 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea714c-3e6e-4a89-9887-986e9e306f92 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:24.159234 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecde9b4-7a01-467f-8d4a-31533d5aef37 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:24.196038 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b575a9c-945a-4e03-a8db-8a605a8e5760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:24.197209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b575a9c-945a-4e03-a8db-8a605a8e5760 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:24.197686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b575a9c-945a-4e03-a8db-8a605a8e5760 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:24.198253 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b575a9c-945a-4e03-a8db-8a605a8e5760 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:24.203685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:24.211431 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b575a9c-945a-4e03-a8db-8a605a8e5760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:24.219719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-0b575a9c-945a-4e03-a8db-8a605a8e5760 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:24.219719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:24.219719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:24.219719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:24.219719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:24.219719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:24.219719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:24.219719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:24.219719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:24.219719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:24.219719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:24.219719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:24.219719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:24.219719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:24.219719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:24.219719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:24.220599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:24.220599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:24.220599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:24.220599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:24.220599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:24.220599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:24.220599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:24.220599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:24.220599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:24.220599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:24.220599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:24.220599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:24.220599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:24.220599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:24.220599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:24.220599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:24.221405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:24.221405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:24.221405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:24.221405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:24.221405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:24.221405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:24.221405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:24.221405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:24.221405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:24.221405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:24.221405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:24.221405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:24.224557 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 101/213] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:28:24.233451 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b575a9c-945a-4e03-a8db-8a605a8e5760 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:24.233604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea714c-3e6e-4a89-9887-986e9e306f92 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:24.268624 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f6c0d-d745-4886-b843-72a0a11005df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:24.269431 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f6c0d-d745-4886-b843-72a0a11005df None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:24.269764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f6c0d-d745-4886-b843-72a0a11005df None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:24.270072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f6c0d-d745-4886-b843-72a0a11005df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:24.271257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f6c0d-d745-4886-b843-72a0a11005df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:24.272726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f6c0d-d745-4886-b843-72a0a11005df None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:24.279009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 113/214] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:28:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:24.279899 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f6c0d-d745-4886-b843-72a0a11005df None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:24.308878 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:24.344994 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b575a9c-945a-4e03-a8db-8a605a8e5760 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:24.383389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688db69e-831d-4a05-aef4-8ca73cfcc37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:24.383986 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688db69e-831d-4a05-aef4-8ca73cfcc37c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:24.384273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688db69e-831d-4a05-aef4-8ca73cfcc37c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:24.384553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688db69e-831d-4a05-aef4-8ca73cfcc37c None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:24.385114 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f6c0d-d745-4886-b843-72a0a11005df None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:24.385314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688db69e-831d-4a05-aef4-8ca73cfcc37c None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:24.385626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688db69e-831d-4a05-aef4-8ca73cfcc37c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:24.387175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688db69e-831d-4a05-aef4-8ca73cfcc37c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:24.401177 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 102/215] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:24.402673 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688db69e-831d-4a05-aef4-8ca73cfcc37c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:24.422958 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4a83d-f962-4945-a2e0-d0929d3080d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:24.423684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4a83d-f962-4945-a2e0-d0929d3080d9 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:24.423973 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4a83d-f962-4945-a2e0-d0929d3080d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:24.424248 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4a83d-f962-4945-a2e0-d0929d3080d9 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:24.424959 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4a83d-f962-4945-a2e0-d0929d3080d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:24.426464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4a83d-f962-4945-a2e0-d0929d3080d9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:24.442529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:24.450449 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 114/216] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:28:24 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:28:24.466857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200fdf7-bc65-487f-8a64-f9a94c4e3c76 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:24.467295 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200fdf7-bc65-487f-8a64-f9a94c4e3c76 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:24.467670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200fdf7-bc65-487f-8a64-f9a94c4e3c76 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:24.469775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 115/217] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:28:24.475642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba823a2f-7499-4d32-ae4f-ab2b0ba192f0 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:24.476213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba823a2f-7499-4d32-ae4f-ab2b0ba192f0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:24.476698 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba823a2f-7499-4d32-ae4f-ab2b0ba192f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:24.499012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ba823a2f-7499-4d32-ae4f-ab2b0ba192f0 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:24.500328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ba823a2f-7499-4d32-ae4f-ab2b0ba192f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:24.544018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688db69e-831d-4a05-aef4-8ca73cfcc37c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:24.588459 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:24.596257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e754403-56d4-4994-9903-385bd9541428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:24.602010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 116/218] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:24.603242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e754403-56d4-4994-9903-385bd9541428 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:24.603534 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e754403-56d4-4994-9903-385bd9541428 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:24.603811 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e754403-56d4-4994-9903-385bd9541428 None admin] PATH_INFO: `/v3/users/37cb64e3970448d7ac84496d1b59dd7d` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:24.624648 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba823a2f-7499-4d32-ae4f-ab2b0ba192f0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:24.625229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e754403-56d4-4994-9903-385bd9541428 None admin] RBAC: Authorizing `identity:get_user(user_id=37cb64e3970448d7ac84496d1b59dd7d)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:24.626941 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e754403-56d4-4994-9903-385bd9541428 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:24.630246 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 103/219] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:24 2022] GET /identity/v3/users/37cb64e3970448d7ac84496d1b59dd7d => generated 297 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:24.635964 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e754403-56d4-4994-9903-385bd9541428 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:24.743952 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba823a2f-7499-4d32-ae4f-ab2b0ba192f0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:24.750257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e754403-56d4-4994-9903-385bd9541428 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:24.773509 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:24.782870 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac6b6d-547f-488a-9268-9a1f214cecf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:24.783684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac6b6d-547f-488a-9268-9a1f214cecf9 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:24.783970 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac6b6d-547f-488a-9268-9a1f214cecf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:24.784245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac6b6d-547f-488a-9268-9a1f214cecf9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:24.797074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b20e8-d3d4-41bd-bf9b-4fe2de7e2d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:24.797740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b20e8-d3d4-41bd-bf9b-4fe2de7e2d92 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:24.798052 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b20e8-d3d4-41bd-bf9b-4fe2de7e2d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:24.798328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b20e8-d3d4-41bd-bf9b-4fe2de7e2d92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:24.801344 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac6b6d-547f-488a-9268-9a1f214cecf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:24.811758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-d2ac6b6d-547f-488a-9268-9a1f214cecf9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:24.811758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:24.811758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:24.811758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:24.811758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:24.811758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:24.811758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:24.811758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:24.811758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:24.811758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:24.811758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:24.811758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:24.811758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:24.811758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:24.811758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:24.811758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:24.812705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:24.812705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:24.812705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:24.812705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:24.812705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:24.812705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:24.812705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:24.812705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:24.812705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:24.812705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:24.812705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:24.812705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:24.812705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:24.812705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:24.812705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:24.812705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:24.813514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:24.813514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:24.813514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:24.813514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:24.813514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:24.813514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:24.813514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:24.813514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:24.813514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:24.813514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:24.813514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:24.813514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:24.815060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 117/220] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:28:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:28:24.816350 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78c354-f604-4041-846e-881dfe3e9a6e None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:24.816634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78c354-f604-4041-846e-881dfe3e9a6e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:24.816906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78c354-f604-4041-846e-881dfe3e9a6e None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:24.818311 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 118/221] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:24.823349 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78c354-f604-4041-846e-881dfe3e9a6e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:24.874675 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:24.888301 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b20e8-d3d4-41bd-bf9b-4fe2de7e2d92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:24.890949 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b20e8-d3d4-41bd-bf9b-4fe2de7e2d92 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:24.949657 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78c354-f604-4041-846e-881dfe3e9a6e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:24.986797 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19796897-881c-49de-802f-3d7c8d9e472b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:24.987474 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19796897-881c-49de-802f-3d7c8d9e472b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:24.987759 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19796897-881c-49de-802f-3d7c8d9e472b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:24.988033 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19796897-881c-49de-802f-3d7c8d9e472b None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:24.988800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19796897-881c-49de-802f-3d7c8d9e472b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:24.989578 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19796897-881c-49de-802f-3d7c8d9e472b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:24.990706 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19796897-881c-49de-802f-3d7c8d9e472b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:25.000189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 119/222] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:25.002529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72f24a-88ed-4612-97d9-7f72bf33aba2 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:25.003027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72f24a-88ed-4612-97d9-7f72bf33aba2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:25.003747 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72f24a-88ed-4612-97d9-7f72bf33aba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:25.020124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 104/223] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:25.021688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b20e8-d3d4-41bd-bf9b-4fe2de7e2d92 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:25.025809 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-dc72f24a-88ed-4612-97d9-7f72bf33aba2 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:25.026605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-dc72f24a-88ed-4612-97d9-7f72bf33aba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:25.076445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:25.098246 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 120/224] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:25.099102 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72f24a-88ed-4612-97d9-7f72bf33aba2 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:25.121460 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:25.133820 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b20e8-d3d4-41bd-bf9b-4fe2de7e2d92 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:25.173209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e07a89-78f8-4237-9030-f0fa859621eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:25.174378 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e07a89-78f8-4237-9030-f0fa859621eb None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:25.174904 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e07a89-78f8-4237-9030-f0fa859621eb None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:25.175324 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e07a89-78f8-4237-9030-f0fa859621eb None admin] PATH_INFO: `/v3/users/b000c41b72b14a63999b9a2e3e47f989` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:25.176713 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e07a89-78f8-4237-9030-f0fa859621eb None admin] RBAC: Authorizing `identity:get_user(user_id=b000c41b72b14a63999b9a2e3e47f989)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:25.178562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e07a89-78f8-4237-9030-f0fa859621eb None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:25.180762 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 105/225] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:25 2022] GET /identity/v3/users/b000c41b72b14a63999b9a2e3e47f989 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:25.181844 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e07a89-78f8-4237-9030-f0fa859621eb None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:25.222998 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72f24a-88ed-4612-97d9-7f72bf33aba2 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:25.271975 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5434c9-d53c-4abb-b202-5c14fb41d880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:25.272852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5434c9-d53c-4abb-b202-5c14fb41d880 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:25.273134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5434c9-d53c-4abb-b202-5c14fb41d880 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:25.273406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5434c9-d53c-4abb-b202-5c14fb41d880 None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:25.274622 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5434c9-d53c-4abb-b202-5c14fb41d880 None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:25.276105 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5434c9-d53c-4abb-b202-5c14fb41d880 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:25.278043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 121/226] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:25 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:25.278871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5434c9-d53c-4abb-b202-5c14fb41d880 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:25.312731 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e07a89-78f8-4237-9030-f0fa859621eb None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:25.347141 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:25.361873 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8cce49-bb6d-43e0-8322-0ab7479d475c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:25.363632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cce49-bb6d-43e0-8322-0ab7479d475c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:25.364093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cce49-bb6d-43e0-8322-0ab7479d475c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:25.364526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cce49-bb6d-43e0-8322-0ab7479d475c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:25.379841 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8cce49-bb6d-43e0-8322-0ab7479d475c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:25.390902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-dc8cce49-bb6d-43e0-8322-0ab7479d475c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:25.390902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:25.390902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:25.390902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:25.390902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:25.390902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:25.390902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:25.390902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:25.390902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:25.390902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:25.390902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:25.390902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:25.390902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:25.390902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:25.390902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:25.390902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:25.391952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:25.391952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:25.391952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:25.391952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:25.391952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:25.391952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:25.391952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:25.391952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:25.391952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:25.391952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:25.391952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:25.391952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:25.391952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:25.391952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:25.391952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:25.391952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:25.392835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:25.392835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:25.392835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:25.392835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:25.392835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:25.392835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:25.392835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:25.392835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:25.392835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:25.392835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:25.392835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:25.392835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:25.395602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 106/227] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:28:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:25.396769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8cce49-bb6d-43e0-8322-0ab7479d475c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:25.418383 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5434c9-d53c-4abb-b202-5c14fb41d880 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:25.422434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:25.458686 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46064cc8-ba59-41b0-8076-1fee01fc8071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:25.460034 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46064cc8-ba59-41b0-8076-1fee01fc8071 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:25.460513 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46064cc8-ba59-41b0-8076-1fee01fc8071 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:25.460994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46064cc8-ba59-41b0-8076-1fee01fc8071 None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:25.462581 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46064cc8-ba59-41b0-8076-1fee01fc8071 None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:25.464301 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46064cc8-ba59-41b0-8076-1fee01fc8071 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:25.466677 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 122/228] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:25 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:25.467638 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46064cc8-ba59-41b0-8076-1fee01fc8071 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:25.530395 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8cce49-bb6d-43e0-8322-0ab7479d475c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:25.568685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27b0dd-48ce-4ea3-8e98-38d6f0dd136c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:25.569352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27b0dd-48ce-4ea3-8e98-38d6f0dd136c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:25.569647 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27b0dd-48ce-4ea3-8e98-38d6f0dd136c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:25.569914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27b0dd-48ce-4ea3-8e98-38d6f0dd136c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:25.570797 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27b0dd-48ce-4ea3-8e98-38d6f0dd136c None admin] RBAC: Adding query filter params (role.id=ed842da869724411a5b355cd12f1c5c8, scope.project.id=ae719f7141f34603811594a41497e87c, user.id=37cb64e3970448d7ac84496d1b59dd7d) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:25.571115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27b0dd-48ce-4ea3-8e98-38d6f0dd136c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:25.572904 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27b0dd-48ce-4ea3-8e98-38d6f0dd136c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:25.582073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46064cc8-ba59-41b0-8076-1fee01fc8071 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:25.586130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 107/229] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:25 2022] GET /identity/v3/role_assignments?user.id=37cb64e3970448d7ac84496d1b59dd7d&scope.project.id=ae719f7141f34603811594a41497e87c&role.id=ed842da869724411a5b355cd12f1c5c8 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:25.587244 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27b0dd-48ce-4ea3-8e98-38d6f0dd136c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:25.602371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:25.634450 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b323cf-e7a5-4975-a266-8d12b3e71dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:25.635585 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b323cf-e7a5-4975-a266-8d12b3e71dcf None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:25.637428 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b323cf-e7a5-4975-a266-8d12b3e71dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:25.638038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b323cf-e7a5-4975-a266-8d12b3e71dcf None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:25.639027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b323cf-e7a5-4975-a266-8d12b3e71dcf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:25.641054 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b323cf-e7a5-4975-a266-8d12b3e71dcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:25.642882 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b323cf-e7a5-4975-a266-8d12b3e71dcf None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:25.660889 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 123/230] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:28:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:25.670343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b323cf-e7a5-4975-a266-8d12b3e71dcf None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:25.781796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:25.806850 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27b0dd-48ce-4ea3-8e98-38d6f0dd136c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:25.838796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b323cf-e7a5-4975-a266-8d12b3e71dcf None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:25.849782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b7618-e36d-4ffe-836a-55569a169e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:25.849782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b7618-e36d-4ffe-836a-55569a169e8c None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:25.849782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b7618-e36d-4ffe-836a-55569a169e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:25.849782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b7618-e36d-4ffe-836a-55569a169e8c None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c/users/b000c41b72b14a63999b9a2e3e47f989/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:25.856723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b7618-e36d-4ffe-836a-55569a169e8c None admin] RBAC: Authorizing `identity:create_grant(project_id=ae719f7141f34603811594a41497e87c, user_id=b000c41b72b14a63999b9a2e3e47f989, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:25.856723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b7618-e36d-4ffe-836a-55569a169e8c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:25.874905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 108/231] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:28:25 2022] PUT /identity/v3/projects/ae719f7141f34603811594a41497e87c/users/b000c41b72b14a63999b9a2e3e47f989/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:25.905367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22131f0-90b9-4c36-bc74-5891f5ced5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:25.906862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22131f0-90b9-4c36-bc74-5891f5ced5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:25.907148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22131f0-90b9-4c36-bc74-5891f5ced5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:25.907518 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22131f0-90b9-4c36-bc74-5891f5ced5e3 None admin] PATH_INFO: `/v3/users/b000c41b72b14a63999b9a2e3e47f989` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:25.914990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22131f0-90b9-4c36-bc74-5891f5ced5e3 None admin] RBAC: Authorizing `identity:get_user(user_id=b000c41b72b14a63999b9a2e3e47f989)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:25.916515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22131f0-90b9-4c36-bc74-5891f5ced5e3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:25.918710 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 124/232] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:25 2022] GET /identity/v3/users/b000c41b72b14a63999b9a2e3e47f989 => generated 283 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:25.924098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b7618-e36d-4ffe-836a-55569a169e8c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:26.065633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b7618-e36d-4ffe-836a-55569a169e8c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:26.102187 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a3311-0721-468e-bda0-b3b751a5e303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:26.103424 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a3311-0721-468e-bda0-b3b751a5e303 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:26.103943 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a3311-0721-468e-bda0-b3b751a5e303 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:26.104426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a3311-0721-468e-bda0-b3b751a5e303 None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:26.105831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a3311-0721-468e-bda0-b3b751a5e303 None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:26.107634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a3311-0721-468e-bda0-b3b751a5e303 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:26.109941 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 109/233] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:25 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:26.115214 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22131f0-90b9-4c36-bc74-5891f5ced5e3 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:26.181313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:26.249201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22131f0-90b9-4c36-bc74-5891f5ced5e3 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:26.290395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91644e-717c-4d59-8e7d-63d30cb5efa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:26.291295 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91644e-717c-4d59-8e7d-63d30cb5efa5 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:26.291575 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91644e-717c-4d59-8e7d-63d30cb5efa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:26.291844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91644e-717c-4d59-8e7d-63d30cb5efa5 None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c/users/b000c41b72b14a63999b9a2e3e47f989/roles/dbe30950eded4c9ab2bcc0f7545b9bae` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:26.294096 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91644e-717c-4d59-8e7d-63d30cb5efa5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae719f7141f34603811594a41497e87c, user_id=b000c41b72b14a63999b9a2e3e47f989, role_id=dbe30950eded4c9ab2bcc0f7545b9bae)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:26.295577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91644e-717c-4d59-8e7d-63d30cb5efa5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:26.321669 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 125/234] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:28:26 2022] PUT /identity/v3/projects/ae719f7141f34603811594a41497e87c/users/b000c41b72b14a63999b9a2e3e47f989/roles/dbe30950eded4c9ab2bcc0f7545b9bae => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:26.801650 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c0419-eb63-4850-9252-3623ab528edc None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:26.802399 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c0419-eb63-4850-9252-3623ab528edc None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:26.803245 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c0419-eb63-4850-9252-3623ab528edc None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:26.804841 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 110/235] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:26.810446 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375dd16-bbed-49a8-9ed7-48ca54f8e512 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:26.810736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375dd16-bbed-49a8-9ed7-48ca54f8e512 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:26.811017 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375dd16-bbed-49a8-9ed7-48ca54f8e512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:26.828053 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9375dd16-bbed-49a8-9ed7-48ca54f8e512 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:26.829245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9375dd16-bbed-49a8-9ed7-48ca54f8e512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:26.920909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 126/236] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:26.955825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c0419-eb63-4850-9252-3623ab528edc None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:26.976696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:27.094491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c0419-eb63-4850-9252-3623ab528edc None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:27.120943 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233d21c-73fd-4475-b914-908872fb04cd None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:27.121464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233d21c-73fd-4475-b914-908872fb04cd None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:27.121554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233d21c-73fd-4475-b914-908872fb04cd None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:27.123229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 127/237] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:27.128321 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fc3a5-24b0-4922-bab5-3ba1a9fcac8e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:27.128614 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fc3a5-24b0-4922-bab5-3ba1a9fcac8e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:27.128894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fc3a5-24b0-4922-bab5-3ba1a9fcac8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:27.153729 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6a798-1ac8-4f73-80d2-f757a41f3bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:27.154572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6a798-1ac8-4f73-80d2-f757a41f3bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:27.154859 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6a798-1ac8-4f73-80d2-f757a41f3bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:27.155140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6a798-1ac8-4f73-80d2-f757a41f3bc1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:27.158351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-170fc3a5-24b0-4922-bab5-3ba1a9fcac8e None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:27.159771 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-170fc3a5-24b0-4922-bab5-3ba1a9fcac8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:27.178313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6a798-1ac8-4f73-80d2-f757a41f3bc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:27.187623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-92f6a798-1ac8-4f73-80d2-f757a41f3bc1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:27.187623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:27.187623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:27.187623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:27.187623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:27.187623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:27.187623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:27.187623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:27.187623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:27.187623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:27.187623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:27.187623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:27.187623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:27.187623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:27.187623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:27.187623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:27.188608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:27.188608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:27.188608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:27.188608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:27.188608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:27.188608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:27.188608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:27.188608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:27.188608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:27.188608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:27.188608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:27.188608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:27.188608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:27.188608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:27.188608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:27.188608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:27.189454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:27.189454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:27.189454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:27.189454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:27.189454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:27.189454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:27.189454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:27.189454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:27.189454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:27.189454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:27.189454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:27.189454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:27.189454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 111/238] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:27.197150 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6a798-1ac8-4f73-80d2-f757a41f3bc1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:27.202870 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:27.247870 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 128/239] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:27.284478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fc3a5-24b0-4922-bab5-3ba1a9fcac8e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:27.298075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:27.331887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6a798-1ac8-4f73-80d2-f757a41f3bc1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:27.369840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39bf4fe-48c3-4fe8-9433-d8a1aaf7ba06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:27.370930 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39bf4fe-48c3-4fe8-9433-d8a1aaf7ba06 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:27.371418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39bf4fe-48c3-4fe8-9433-d8a1aaf7ba06 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:27.371893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39bf4fe-48c3-4fe8-9433-d8a1aaf7ba06 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:27.372862 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39bf4fe-48c3-4fe8-9433-d8a1aaf7ba06 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:27.373380 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39bf4fe-48c3-4fe8-9433-d8a1aaf7ba06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:27.375044 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39bf4fe-48c3-4fe8-9433-d8a1aaf7ba06 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:27.384288 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 112/240] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:27.390308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39bf4fe-48c3-4fe8-9433-d8a1aaf7ba06 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:27.475955 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fc3a5-24b0-4922-bab5-3ba1a9fcac8e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:27.499900 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39bf4fe-48c3-4fe8-9433-d8a1aaf7ba06 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:27.537442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6124e75-f32f-4498-8cd2-6359bd6ed4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:27.538768 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6124e75-f32f-4498-8cd2-6359bd6ed4bf None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:27.539330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6124e75-f32f-4498-8cd2-6359bd6ed4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:27.539801 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6124e75-f32f-4498-8cd2-6359bd6ed4bf None admin] PATH_INFO: `/v3/users/563026d078554a48a64feb283b6d26de` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:27.554421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:27.559181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6124e75-f32f-4498-8cd2-6359bd6ed4bf None admin] RBAC: Authorizing `identity:get_user(user_id=563026d078554a48a64feb283b6d26de)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:27.563205 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6124e75-f32f-4498-8cd2-6359bd6ed4bf None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:27.565067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 113/241] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:27 2022] GET /identity/v3/users/563026d078554a48a64feb283b6d26de => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:27.573271 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce56c46-9bee-4c44-9bb3-8049521856a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:27.574233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce56c46-9bee-4c44-9bb3-8049521856a1 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:27.574519 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce56c46-9bee-4c44-9bb3-8049521856a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:27.574797 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce56c46-9bee-4c44-9bb3-8049521856a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:27.575011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6124e75-f32f-4498-8cd2-6359bd6ed4bf None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:27.576078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce56c46-9bee-4c44-9bb3-8049521856a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:27.578288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce56c46-9bee-4c44-9bb3-8049521856a1 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:27.580120 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 129/242] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:28:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:27.586103 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce56c46-9bee-4c44-9bb3-8049521856a1 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:27.625776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:27.702087 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce56c46-9bee-4c44-9bb3-8049521856a1 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:27.765793 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f68ee-52e9-47a5-8f95-4f5dee07971d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:27.767023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f68ee-52e9-47a5-8f95-4f5dee07971d None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:27.767309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f68ee-52e9-47a5-8f95-4f5dee07971d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:27.767586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f68ee-52e9-47a5-8f95-4f5dee07971d None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:27.768461 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f68ee-52e9-47a5-8f95-4f5dee07971d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:27.770124 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f68ee-52e9-47a5-8f95-4f5dee07971d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:27.793843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 130/243] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:28:27 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:28:27.883531 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6124e75-f32f-4498-8cd2-6359bd6ed4bf None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:27.938102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207160d-c3db-4542-bcc5-965ea27e7ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:27.938785 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207160d-c3db-4542-bcc5-965ea27e7ecd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:27.939066 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207160d-c3db-4542-bcc5-965ea27e7ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:27.939342 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207160d-c3db-4542-bcc5-965ea27e7ecd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.007067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf46590-df29-4f5f-bd09-2a32bc5e8494 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.007392 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf46590-df29-4f5f-bd09-2a32bc5e8494 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.007635 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf46590-df29-4f5f-bd09-2a32bc5e8494 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.009312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 131/244] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:28:28.015200 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94f5b3-9dde-4da9-9d48-6c5341b895a8 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.015496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94f5b3-9dde-4da9-9d48-6c5341b895a8 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.015759 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94f5b3-9dde-4da9-9d48-6c5341b895a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.019478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:28.035653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207160d-c3db-4542-bcc5-965ea27e7ecd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:28.037394 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207160d-c3db-4542-bcc5-965ea27e7ecd None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:28.041042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-bc94f5b3-9dde-4da9-9d48-6c5341b895a8 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:28.042662 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-bc94f5b3-9dde-4da9-9d48-6c5341b895a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:28.132248 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 132/245] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:28.159992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94f5b3-9dde-4da9-9d48-6c5341b895a8 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.186852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:28.204934 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 114/246] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:28.210255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207160d-c3db-4542-bcc5-965ea27e7ecd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.286225 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94f5b3-9dde-4da9-9d48-6c5341b895a8 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:28.311657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:28.323204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207160d-c3db-4542-bcc5-965ea27e7ecd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:28.327676 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8fcad-c710-4241-b042-729ee470d703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.328557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8fcad-c710-4241-b042-729ee470d703 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.328847 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8fcad-c710-4241-b042-729ee470d703 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.329127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8fcad-c710-4241-b042-729ee470d703 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.342079 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8fcad-c710-4241-b042-729ee470d703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:28.351201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-94d8fcad-c710-4241-b042-729ee470d703 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:28.351201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:28.351201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:28.351201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:28.351201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:28.351201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:28.351201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:28.351201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:28.351201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:28.351201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:28.351201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:28.351201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:28.351201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:28.351201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:28.351201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:28.351201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:28.352126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:28.352126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:28.352126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:28.352126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:28.352126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:28.352126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:28.352126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:28.352126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:28.352126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:28.352126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:28.352126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:28.352126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:28.352126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:28.352126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:28.352126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:28.352126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:28.353001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:28.353001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:28.353001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:28.353001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:28.353001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:28.353001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:28.353001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:28.353001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:28.353001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:28.353001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:28.353001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:28.353001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:28.353001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 133/247] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:28.359749 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8fcad-c710-4241-b042-729ee470d703 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.368526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee509a4-8f6e-4f73-9180-7bc799dcbe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.369753 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee509a4-8f6e-4f73-9180-7bc799dcbe47 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.370265 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee509a4-8f6e-4f73-9180-7bc799dcbe47 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.370757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee509a4-8f6e-4f73-9180-7bc799dcbe47 None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.372177 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee509a4-8f6e-4f73-9180-7bc799dcbe47 None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:28.373923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee509a4-8f6e-4f73-9180-7bc799dcbe47 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:28.376335 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 115/248] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:28 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:28.382180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee509a4-8f6e-4f73-9180-7bc799dcbe47 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.454797 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:28.482333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8fcad-c710-4241-b042-729ee470d703 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:28.502680 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee509a4-8f6e-4f73-9180-7bc799dcbe47 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:28.528965 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bd3ce-9aa8-4431-8d30-7040088dea9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.528965 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bd3ce-9aa8-4431-8d30-7040088dea9f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.528965 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bd3ce-9aa8-4431-8d30-7040088dea9f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.528965 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bd3ce-9aa8-4431-8d30-7040088dea9f None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.529478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bd3ce-9aa8-4431-8d30-7040088dea9f None admin] RBAC: Adding query filter params (name=member) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:28.529856 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bd3ce-9aa8-4431-8d30-7040088dea9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:28.532214 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bd3ce-9aa8-4431-8d30-7040088dea9f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:28.545761 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 134/249] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:28.547525 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbfbfd-33b4-427c-be41-2b96f81a18f4 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.547997 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbfbfd-33b4-427c-be41-2b96f81a18f4 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.548475 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbfbfd-33b4-427c-be41-2b96f81a18f4 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.554460 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 135/250] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:28.555736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbfbfd-33b4-427c-be41-2b96f81a18f4 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.558554 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe7a14-880e-462a-94cc-77920d96104d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.559756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe7a14-880e-462a-94cc-77920d96104d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.561174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe7a14-880e-462a-94cc-77920d96104d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.561700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe7a14-880e-462a-94cc-77920d96104d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.562811 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe7a14-880e-462a-94cc-77920d96104d None admin] RBAC: Adding query filter params (role.id=3d88c071954947f99ede3e31fb58494a, scope.project.id=ae719f7141f34603811594a41497e87c, user.id=563026d078554a48a64feb283b6d26de) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:28.563335 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe7a14-880e-462a-94cc-77920d96104d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:28.565014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe7a14-880e-462a-94cc-77920d96104d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:28.572570 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 116/251] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:28 2022] GET /identity/v3/role_assignments?user.id=563026d078554a48a64feb283b6d26de&scope.project.id=ae719f7141f34603811594a41497e87c&role.id=3d88c071954947f99ede3e31fb58494a => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:28.574292 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60128ad1-f885-4940-81df-873c0121375e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.574737 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60128ad1-f885-4940-81df-873c0121375e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.575634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60128ad1-f885-4940-81df-873c0121375e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.598807 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-60128ad1-f885-4940-81df-873c0121375e None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:28.600761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-60128ad1-f885-4940-81df-873c0121375e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:28.626532 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:28.690392 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbfbfd-33b4-427c-be41-2b96f81a18f4 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:28.692160 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 117/252] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:28.744743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6e12c-3079-4fc3-bc18-5793d2327f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.745998 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6e12c-3079-4fc3-bc18-5793d2327f63 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.746460 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6e12c-3079-4fc3-bc18-5793d2327f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.746882 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6e12c-3079-4fc3-bc18-5793d2327f63 None admin] PATH_INFO: `/v3/users/37cb64e3970448d7ac84496d1b59dd7d` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.748937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6e12c-3079-4fc3-bc18-5793d2327f63 None admin] RBAC: Authorizing `identity:get_user(user_id=37cb64e3970448d7ac84496d1b59dd7d)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:28.750722 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6e12c-3079-4fc3-bc18-5793d2327f63 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:28.752347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60128ad1-f885-4940-81df-873c0121375e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.755170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 136/253] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:28 2022] GET /identity/v3/users/37cb64e3970448d7ac84496d1b59dd7d => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:28.760191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6e12c-3079-4fc3-bc18-5793d2327f63 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.783852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:28.946976 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6e12c-3079-4fc3-bc18-5793d2327f63 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:29.000573 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60128ad1-f885-4940-81df-873c0121375e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:29.005998 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0570dc-75c3-4f45-8150-0b16a03c41ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:29.006899 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0570dc-75c3-4f45-8150-0b16a03c41ee None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:29.007184 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0570dc-75c3-4f45-8150-0b16a03c41ee None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:29.007604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0570dc-75c3-4f45-8150-0b16a03c41ee None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.012400 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0570dc-75c3-4f45-8150-0b16a03c41ee None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:29.013885 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0570dc-75c3-4f45-8150-0b16a03c41ee None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:29.016237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 137/254] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:28 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:29.022147 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0570dc-75c3-4f45-8150-0b16a03c41ee None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:29.073535 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446847d8-b7b1-4781-9c54-c11585018ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:29.074372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446847d8-b7b1-4781-9c54-c11585018ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:29.074652 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446847d8-b7b1-4781-9c54-c11585018ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:29.074928 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446847d8-b7b1-4781-9c54-c11585018ab8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.103213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446847d8-b7b1-4781-9c54-c11585018ab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:29.111909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-446847d8-b7b1-4781-9c54-c11585018ab8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:29.111909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:29.111909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:29.111909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:29.111909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:29.111909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:29.111909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:29.111909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:29.111909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:29.111909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:29.111909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:29.111909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:29.111909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:29.111909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:29.111909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:29.111909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:29.112809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:29.112809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:29.112809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:29.112809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:29.112809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:29.112809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:29.112809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:29.112809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:29.112809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:29.112809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:29.112809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:29.112809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:29.112809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:29.112809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:29.112809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:29.112809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:29.114147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:29.114147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:29.114147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:29.114147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:29.114147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:29.114147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:29.114147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:29.114147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:29.114147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:29.114147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:29.114147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:29.114147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:29.114147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 118/255] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:28:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:28:29.119241 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446847d8-b7b1-4781-9c54-c11585018ab8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:29.143750 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:29.205151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:29.227479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0570dc-75c3-4f45-8150-0b16a03c41ee None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:29.293216 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70124e71-a69e-4bd6-8fee-62d08b181457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:29.294059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70124e71-a69e-4bd6-8fee-62d08b181457 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:29.294344 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70124e71-a69e-4bd6-8fee-62d08b181457 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:29.294625 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70124e71-a69e-4bd6-8fee-62d08b181457 None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c/users/37cb64e3970448d7ac84496d1b59dd7d/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.298326 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70124e71-a69e-4bd6-8fee-62d08b181457 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae719f7141f34603811594a41497e87c, user_id=37cb64e3970448d7ac84496d1b59dd7d, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:29.299802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70124e71-a69e-4bd6-8fee-62d08b181457 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:29.310209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446847d8-b7b1-4781-9c54-c11585018ab8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:29.324266 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 138/256] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:28:29 2022] PUT /identity/v3/projects/ae719f7141f34603811594a41497e87c/users/37cb64e3970448d7ac84496d1b59dd7d/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:29.325692 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1b577-e664-4d7b-b6dc-155417c45fb7 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:29.327268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1b577-e664-4d7b-b6dc-155417c45fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:29.327558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1b577-e664-4d7b-b6dc-155417c45fb7 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.332671 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 139/257] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:29.339264 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd596b-9be3-469b-b205-40dd43a835b5 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:29.339545 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd596b-9be3-469b-b205-40dd43a835b5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:29.339829 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd596b-9be3-469b-b205-40dd43a835b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.373020 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-cccd596b-9be3-469b-b205-40dd43a835b5 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:29.374864 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cccd596b-9be3-469b-b205-40dd43a835b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:29.381222 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee55519-54f3-403e-acc8-5098395a89a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:29.381900 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee55519-54f3-403e-acc8-5098395a89a9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:29.382226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee55519-54f3-403e-acc8-5098395a89a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:29.382506 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee55519-54f3-403e-acc8-5098395a89a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.383274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee55519-54f3-403e-acc8-5098395a89a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:29.383595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee55519-54f3-403e-acc8-5098395a89a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:29.385033 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee55519-54f3-403e-acc8-5098395a89a9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:29.402414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 119/258] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:29.410511 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee55519-54f3-403e-acc8-5098395a89a9 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:29.559895 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:29.561591 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 140/259] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:29.596854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee55519-54f3-403e-acc8-5098395a89a9 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:29.598253 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd596b-9be3-469b-b205-40dd43a835b5 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:29.634787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a0750-ae26-41b5-878e-5409b1ab0358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:29.635673 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a0750-ae26-41b5-878e-5409b1ab0358 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:29.635955 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a0750-ae26-41b5-878e-5409b1ab0358 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:29.636227 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a0750-ae26-41b5-878e-5409b1ab0358 None admin] PATH_INFO: `/v3/users/b000c41b72b14a63999b9a2e3e47f989` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.637445 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a0750-ae26-41b5-878e-5409b1ab0358 None admin] RBAC: Authorizing `identity:get_user(user_id=b000c41b72b14a63999b9a2e3e47f989)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:29.639344 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a0750-ae26-41b5-878e-5409b1ab0358 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:29.640959 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 120/260] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:29 2022] GET /identity/v3/users/b000c41b72b14a63999b9a2e3e47f989 => generated 283 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:29.646259 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a0750-ae26-41b5-878e-5409b1ab0358 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:29.708045 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:29.761761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a0750-ae26-41b5-878e-5409b1ab0358 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:29.765793 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd596b-9be3-469b-b205-40dd43a835b5 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:29.799028 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd269d81-5d14-491a-8564-bd64d246e281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:29.799693 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd269d81-5d14-491a-8564-bd64d246e281 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:29.799978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd269d81-5d14-491a-8564-bd64d246e281 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:29.800256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd269d81-5d14-491a-8564-bd64d246e281 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.815020 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:29.834682 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2ef36-55dd-44f1-855a-d88e2ffde793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:29.835853 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2ef36-55dd-44f1-855a-d88e2ffde793 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:29.836343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2ef36-55dd-44f1-855a-d88e2ffde793 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:29.836820 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2ef36-55dd-44f1-855a-d88e2ffde793 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.855371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2ef36-55dd-44f1-855a-d88e2ffde793 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:29.866605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-ebf2ef36-55dd-44f1-855a-d88e2ffde793 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:29.866605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:29.866605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:29.866605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:29.866605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:29.866605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:29.866605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:29.866605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:29.866605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:29.866605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:29.866605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:29.866605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:29.866605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:29.866605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:29.866605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:29.866605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:29.868328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:29.868328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:29.868328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:29.868328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:29.868328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:29.868328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:29.868328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:29.868328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:29.868328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:29.868328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:29.868328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:29.868328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:29.868328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:29.868328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:29.868328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:29.868328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:29.869894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:29.869894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:29.869894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:29.869894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:29.869894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:29.869894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:29.869894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:29.869894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:29.869894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:29.869894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:29.869894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:29.869894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:29.879047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 141/261] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:28:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:29.884263 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2ef36-55dd-44f1-855a-d88e2ffde793 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:29.911022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd269d81-5d14-491a-8564-bd64d246e281 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:29.913023 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd269d81-5d14-491a-8564-bd64d246e281 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.048480 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 121/262] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 13:28:30.055789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd269d81-5d14-491a-8564-bd64d246e281 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.059912 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2ef36-55dd-44f1-855a-d88e2ffde793 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.102857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:30.103639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:30.116000 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa838b69-2542-42fa-8595-1ac9f861f2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.116766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa838b69-2542-42fa-8595-1ac9f861f2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.117138 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa838b69-2542-42fa-8595-1ac9f861f2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.117482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa838b69-2542-42fa-8595-1ac9f861f2f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.118339 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa838b69-2542-42fa-8595-1ac9f861f2f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:30.118712 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa838b69-2542-42fa-8595-1ac9f861f2f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.120212 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa838b69-2542-42fa-8595-1ac9f861f2f1 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.137868 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 142/263] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:28:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:30.142912 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa838b69-2542-42fa-8595-1ac9f861f2f1 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.171447 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd269d81-5d14-491a-8564-bd64d246e281 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.209304 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413c313-a4bf-4557-9f58-648927039f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.211037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413c313-a4bf-4557-9f58-648927039f1e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.211501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413c313-a4bf-4557-9f58-648927039f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.211951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413c313-a4bf-4557-9f58-648927039f1e None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.213358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413c313-a4bf-4557-9f58-648927039f1e None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.215682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413c313-a4bf-4557-9f58-648927039f1e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.218139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 122/264] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:30 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:30.224805 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413c313-a4bf-4557-9f58-648927039f1e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.348159 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa838b69-2542-42fa-8595-1ac9f861f2f1 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.354250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413c313-a4bf-4557-9f58-648927039f1e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.368650 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:30.398644 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede03285-752b-49e9-bed2-2047a02856b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.399722 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede03285-752b-49e9-bed2-2047a02856b1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.400322 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede03285-752b-49e9-bed2-2047a02856b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.400919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede03285-752b-49e9-bed2-2047a02856b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.402510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede03285-752b-49e9-bed2-2047a02856b1 None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=ae719f7141f34603811594a41497e87c, user.id=b000c41b72b14a63999b9a2e3e47f989) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:30.402995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede03285-752b-49e9-bed2-2047a02856b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.404602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede03285-752b-49e9-bed2-2047a02856b1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.412126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 123/265] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:30 2022] GET /identity/v3/role_assignments?user.id=b000c41b72b14a63999b9a2e3e47f989&scope.project.id=ae719f7141f34603811594a41497e87c&role.id=864a6966630544e8bd68bcafba905c6f => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:30.444315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45411644-de37-4abb-a933-e4009722f68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.445225 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45411644-de37-4abb-a933-e4009722f68f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.445508 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45411644-de37-4abb-a933-e4009722f68f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.445798 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45411644-de37-4abb-a933-e4009722f68f None admin] PATH_INFO: `/v3/users/b000c41b72b14a63999b9a2e3e47f989` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.450935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45411644-de37-4abb-a933-e4009722f68f None admin] RBAC: Authorizing `identity:get_user(user_id=b000c41b72b14a63999b9a2e3e47f989)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.452518 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45411644-de37-4abb-a933-e4009722f68f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.454422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 143/266] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:30 2022] GET /identity/v3/users/b000c41b72b14a63999b9a2e3e47f989 => generated 283 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:30.460182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede03285-752b-49e9-bed2-2047a02856b1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.613342 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede03285-752b-49e9-bed2-2047a02856b1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.659813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191aeef-a234-4053-acf8-f5c1c2c7feb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.660887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191aeef-a234-4053-acf8-f5c1c2c7feb0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.661370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191aeef-a234-4053-acf8-f5c1c2c7feb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.661854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191aeef-a234-4053-acf8-f5c1c2c7feb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.714299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:30.744699 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2191aeef-a234-4053-acf8-f5c1c2c7feb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.746814 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2191aeef-a234-4053-acf8-f5c1c2c7feb0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.868149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 124/267] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:30.873374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45411644-de37-4abb-a933-e4009722f68f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.938341 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:30.983131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45411644-de37-4abb-a933-e4009722f68f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:31.024837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada220f5-ca40-48ef-94ee-5419d0804f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:31.026164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada220f5-ca40-48ef-94ee-5419d0804f4a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.026653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada220f5-ca40-48ef-94ee-5419d0804f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.027127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada220f5-ca40-48ef-94ee-5419d0804f4a None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.028532 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada220f5-ca40-48ef-94ee-5419d0804f4a None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:31.030844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada220f5-ca40-48ef-94ee-5419d0804f4a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:31.033103 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 144/268] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:30 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:31.038544 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191aeef-a234-4053-acf8-f5c1c2c7feb0 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:31.096715 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d0300-a302-4cce-bc1c-e59fd828692a None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.097349 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d0300-a302-4cce-bc1c-e59fd828692a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.097854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d0300-a302-4cce-bc1c-e59fd828692a None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.099712 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 145/269] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:28:31.106761 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ca63a-3b2b-4286-afd7-3562ec61b419 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.107390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ca63a-3b2b-4286-afd7-3562ec61b419 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.107872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ca63a-3b2b-4286-afd7-3562ec61b419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.130828 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-305ca63a-3b2b-4286-afd7-3562ec61b419 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:31.133211 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-305ca63a-3b2b-4286-afd7-3562ec61b419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:31.140500 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:31.173025 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191aeef-a234-4053-acf8-f5c1c2c7feb0 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:31.222091 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64554e2b-8f89-4761-8eb7-15701c348df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:31.222759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64554e2b-8f89-4761-8eb7-15701c348df9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.223038 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64554e2b-8f89-4761-8eb7-15701c348df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.223309 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64554e2b-8f89-4761-8eb7-15701c348df9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.224633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64554e2b-8f89-4761-8eb7-15701c348df9 None admin] RBAC: Adding query filter params (role.id=dbe30950eded4c9ab2bcc0f7545b9bae, scope.project.id=ae719f7141f34603811594a41497e87c, user.id=b000c41b72b14a63999b9a2e3e47f989) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:31.224963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64554e2b-8f89-4761-8eb7-15701c348df9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:31.226579 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64554e2b-8f89-4761-8eb7-15701c348df9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:31.238341 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 125/270] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:31 2022] GET /identity/v3/role_assignments?user.id=b000c41b72b14a63999b9a2e3e47f989&scope.project.id=ae719f7141f34603811594a41497e87c&role.id=dbe30950eded4c9ab2bcc0f7545b9bae => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:31.256757 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 146/271] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:31.282622 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64554e2b-8f89-4761-8eb7-15701c348df9 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:31.430385 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64554e2b-8f89-4761-8eb7-15701c348df9 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:31.462675 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3992ea-314b-47e5-89cb-e09a73e2febb None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.463649 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3992ea-314b-47e5-89cb-e09a73e2febb None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.464144 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3992ea-314b-47e5-89cb-e09a73e2febb None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.465812 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 147/272] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:31.471733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4bab2-f1d8-48de-8339-02c3e954235c None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.472024 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4bab2-f1d8-48de-8339-02c3e954235c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.472297 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4bab2-f1d8-48de-8339-02c3e954235c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.474648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653a877-d237-4e97-a7cf-7df5ecaf6d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:31.475463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653a877-d237-4e97-a7cf-7df5ecaf6d3a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.475745 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653a877-d237-4e97-a7cf-7df5ecaf6d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.476017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653a877-d237-4e97-a7cf-7df5ecaf6d3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.477228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1653a877-d237-4e97-a7cf-7df5ecaf6d3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:31.478763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1653a877-d237-4e97-a7cf-7df5ecaf6d3a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:31.480468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 126/273] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:28:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:31.486174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653a877-d237-4e97-a7cf-7df5ecaf6d3a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:31.493115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-80c4bab2-f1d8-48de-8339-02c3e954235c None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:31.494684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-80c4bab2-f1d8-48de-8339-02c3e954235c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:31.496061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:31.578869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:31.594248 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 148/274] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:31.618608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4bab2-f1d8-48de-8339-02c3e954235c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:31.661539 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653a877-d237-4e97-a7cf-7df5ecaf6d3a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:31.716120 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1521a-2d6a-4dae-832a-f0eca03f8fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:31.716727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1521a-2d6a-4dae-832a-f0eca03f8fb4 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.717014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1521a-2d6a-4dae-832a-f0eca03f8fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.717291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1521a-2d6a-4dae-832a-f0eca03f8fb4 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.718246 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1521a-2d6a-4dae-832a-f0eca03f8fb4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:31.719822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1521a-2d6a-4dae-832a-f0eca03f8fb4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:31.745710 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 127/275] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:28:31 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:31.747435 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad538c59-e27b-41b0-8485-c60f84af2772 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.747782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad538c59-e27b-41b0-8485-c60f84af2772 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.748438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad538c59-e27b-41b0-8485-c60f84af2772 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.750654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 128/276] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:31.756712 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1f802-df64-44e4-a254-9c5f288589b0 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.756994 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1f802-df64-44e4-a254-9c5f288589b0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.757274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1f802-df64-44e4-a254-9c5f288589b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.762759 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4bab2-f1d8-48de-8339-02c3e954235c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:31.775056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-46f1f802-df64-44e4-a254-9c5f288589b0 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:31.776382 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-46f1f802-df64-44e4-a254-9c5f288589b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:31.802630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498af054-f91a-4001-a82f-b9d4190421aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:31.804017 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498af054-f91a-4001-a82f-b9d4190421aa None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.804572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498af054-f91a-4001-a82f-b9d4190421aa None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.804871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498af054-f91a-4001-a82f-b9d4190421aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.821327 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498af054-f91a-4001-a82f-b9d4190421aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:31.832937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-498af054-f91a-4001-a82f-b9d4190421aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:31.832937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:31.832937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:31.832937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:31.832937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:31.832937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:31.832937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:31.832937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:31.832937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:31.832937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:31.832937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:31.832937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:31.832937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:31.832937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:31.832937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:31.832937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:31.833925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:31.833925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:31.833925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:31.833925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:31.833925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:31.833925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:31.833925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:31.833925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:31.833925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:31.833925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:31.833925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:31.833925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:31.833925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:31.833925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:31.833925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:31.833925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:31.834824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:31.834824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:31.834824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:31.834824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:31.834824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:31.834824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:31.834824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:31.834824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:31.834824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:31.834824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:31.834824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:31.834824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:31.834824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 149/277] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:28:31.840947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498af054-f91a-4001-a82f-b9d4190421aa None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:31.848169 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:31.869984 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 129/278] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:31.892669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1f802-df64-44e4-a254-9c5f288589b0 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:31.906326 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:31.974912 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498af054-f91a-4001-a82f-b9d4190421aa None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.021876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82eb3f-5cab-4824-8647-dcca0183ebe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.022357 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1f802-df64-44e4-a254-9c5f288589b0 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.022588 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82eb3f-5cab-4824-8647-dcca0183ebe6 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.022873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82eb3f-5cab-4824-8647-dcca0183ebe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.023148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82eb3f-5cab-4824-8647-dcca0183ebe6 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.024064 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82eb3f-5cab-4824-8647-dcca0183ebe6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:32.024372 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82eb3f-5cab-4824-8647-dcca0183ebe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.026619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82eb3f-5cab-4824-8647-dcca0183ebe6 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.037953 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 150/279] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:32.043390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82eb3f-5cab-4824-8647-dcca0183ebe6 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.064557 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a4ec9-d4cd-485e-ac23-35f0130db881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.065372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a4ec9-d4cd-485e-ac23-35f0130db881 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.065654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a4ec9-d4cd-485e-ac23-35f0130db881 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.065947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a4ec9-d4cd-485e-ac23-35f0130db881 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.081567 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a4ec9-d4cd-485e-ac23-35f0130db881 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:32.089778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-ba4a4ec9-d4cd-485e-ac23-35f0130db881 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:32.089778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:32.089778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:32.089778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:32.089778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:32.089778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:32.089778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:32.089778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:32.089778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:32.089778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:32.089778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:32.089778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:32.089778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:32.089778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:32.089778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:32.089778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:32.090757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:32.090757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:32.090757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:32.090757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:32.090757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:32.090757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:32.090757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:32.090757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:32.090757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:32.090757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:32.090757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:32.090757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:32.090757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:32.090757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:32.090757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:32.090757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:32.091549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:32.091549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:32.091549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:32.091549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:32.091549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:32.091549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:32.091549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:32.091549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:32.091549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:32.091549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:32.091549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:32.091549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:32.091549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 130/280] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:28:32.098328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a4ec9-d4cd-485e-ac23-35f0130db881 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.169632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82eb3f-5cab-4824-8647-dcca0183ebe6 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.173793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:32.188483 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:32.228200 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90318692-f08a-4c1b-be36-38b994dde21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.229419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90318692-f08a-4c1b-be36-38b994dde21b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.229910 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90318692-f08a-4c1b-be36-38b994dde21b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.230442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90318692-f08a-4c1b-be36-38b994dde21b None admin] PATH_INFO: `/v3/users/1651993e33964848973f033abfe8003b` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.240813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a4ec9-d4cd-485e-ac23-35f0130db881 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.248154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90318692-f08a-4c1b-be36-38b994dde21b None admin] RBAC: Authorizing `identity:get_user(user_id=1651993e33964848973f033abfe8003b)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.250165 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90318692-f08a-4c1b-be36-38b994dde21b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.252544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 151/281] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:32 2022] GET /identity/v3/users/1651993e33964848973f033abfe8003b => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:32.262998 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90318692-f08a-4c1b-be36-38b994dde21b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.290823 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc3426-c5e4-46b0-ac97-8b9e6acc646e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.291502 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc3426-c5e4-46b0-ac97-8b9e6acc646e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.291789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc3426-c5e4-46b0-ac97-8b9e6acc646e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.292063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc3426-c5e4-46b0-ac97-8b9e6acc646e None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.293393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc3426-c5e4-46b0-ac97-8b9e6acc646e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:32.293712 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc3426-c5e4-46b0-ac97-8b9e6acc646e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.295434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc3426-c5e4-46b0-ac97-8b9e6acc646e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.308146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 131/282] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:32.316230 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc3426-c5e4-46b0-ac97-8b9e6acc646e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.425834 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90318692-f08a-4c1b-be36-38b994dde21b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.484495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903b9cc-e6eb-4ffe-9061-cecc8e48e389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.485087 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903b9cc-e6eb-4ffe-9061-cecc8e48e389 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.485352 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903b9cc-e6eb-4ffe-9061-cecc8e48e389 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.485600 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903b9cc-e6eb-4ffe-9061-cecc8e48e389 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.486458 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903b9cc-e6eb-4ffe-9061-cecc8e48e389 None admin] RBAC: Adding query filter params (role.id=ed842da869724411a5b355cd12f1c5c8, user.id=1651993e33964848973f033abfe8003b, scope.system=all) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:32.486778 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903b9cc-e6eb-4ffe-9061-cecc8e48e389 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.490072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903b9cc-e6eb-4ffe-9061-cecc8e48e389 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.500309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 152/283] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Jul 12 13:28:32 2022] GET /identity/v3/role_assignments?user.id=1651993e33964848973f033abfe8003b&scope.system=all&role.id=ed842da869724411a5b355cd12f1c5c8 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:32.503336 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc3426-c5e4-46b0-ac97-8b9e6acc646e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.531775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:32.543181 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e79619-6f1a-4a4e-86a3-cbac4ae8dd9f None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.543799 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e79619-6f1a-4a4e-86a3-cbac4ae8dd9f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.544269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e79619-6f1a-4a4e-86a3-cbac4ae8dd9f None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.546003 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 153/284] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:28:32.551391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6319dc-c4fb-452b-80f6-5d5b54d3b72a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.551968 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6319dc-c4fb-452b-80f6-5d5b54d3b72a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.552438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6319dc-c4fb-452b-80f6-5d5b54d3b72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.575496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ca6319dc-c4fb-452b-80f6-5d5b54d3b72a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:32.577782 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ca6319dc-c4fb-452b-80f6-5d5b54d3b72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:32.586283 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ce2c7-6114-4128-b401-dd0c7e8aaa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.586283 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ce2c7-6114-4128-b401-dd0c7e8aaa8d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.586283 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ce2c7-6114-4128-b401-dd0c7e8aaa8d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.586283 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ce2c7-6114-4128-b401-dd0c7e8aaa8d None admin] PATH_INFO: `/v3/users/563026d078554a48a64feb283b6d26de` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.586647 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ce2c7-6114-4128-b401-dd0c7e8aaa8d None admin] RBAC: Authorizing `identity:get_user(user_id=563026d078554a48a64feb283b6d26de)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.587492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ce2c7-6114-4128-b401-dd0c7e8aaa8d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.590460 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 132/285] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:32 2022] GET /identity/v3/users/563026d078554a48a64feb283b6d26de => generated 297 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:32.600845 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ce2c7-6114-4128-b401-dd0c7e8aaa8d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.633174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:32.718336 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 154/286] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:32.755832 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6319dc-c4fb-452b-80f6-5d5b54d3b72a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.795387 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ce2c7-6114-4128-b401-dd0c7e8aaa8d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.864611 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70718461-95b3-4ee3-9d22-244bab05e088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.865671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70718461-95b3-4ee3-9d22-244bab05e088 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.866672 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70718461-95b3-4ee3-9d22-244bab05e088 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.867203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70718461-95b3-4ee3-9d22-244bab05e088 None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.871786 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70718461-95b3-4ee3-9d22-244bab05e088 None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.873345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70718461-95b3-4ee3-9d22-244bab05e088 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.876719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 133/287] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:32 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:32.883990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70718461-95b3-4ee3-9d22-244bab05e088 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.938209 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6319dc-c4fb-452b-80f6-5d5b54d3b72a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.976468 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdc90e-3a05-4b81-89b8-6e3e70ad0364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.977435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdc90e-3a05-4b81-89b8-6e3e70ad0364 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.977791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdc90e-3a05-4b81-89b8-6e3e70ad0364 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.978165 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdc90e-3a05-4b81-89b8-6e3e70ad0364 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.992411 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdc90e-3a05-4b81-89b8-6e3e70ad0364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:33.003422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-1fbdc90e-3a05-4b81-89b8-6e3e70ad0364 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:33.003422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:33.003422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:33.003422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:33.003422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:33.003422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:33.003422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:33.003422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:33.003422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:33.003422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:33.003422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:33.003422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:33.003422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:33.003422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:33.003422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:33.003422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:33.004791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:33.004791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:33.004791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:33.004791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:33.004791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:33.004791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:33.004791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:33.004791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:33.004791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:33.004791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:33.004791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:33.004791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:33.004791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:33.004791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:33.004791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:33.004791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:33.006742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:33.006742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:33.006742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:33.006742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:33.006742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:33.006742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:33.006742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:33.006742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:33.006742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:33.006742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:33.006742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:33.006742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:33.007703 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 155/288] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:33.015481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdc90e-3a05-4b81-89b8-6e3e70ad0364 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:33.018228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:33.075606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70718461-95b3-4ee3-9d22-244bab05e088 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:33.081081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:33.113510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ec660-4dff-48f2-a013-07a130a4f7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:33.114447 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ec660-4dff-48f2-a013-07a130a4f7d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:33.114727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ec660-4dff-48f2-a013-07a130a4f7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:33.115004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ec660-4dff-48f2-a013-07a130a4f7d2 None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c/users/563026d078554a48a64feb283b6d26de/roles/3d88c071954947f99ede3e31fb58494a` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:33.117020 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ec660-4dff-48f2-a013-07a130a4f7d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae719f7141f34603811594a41497e87c, user_id=563026d078554a48a64feb283b6d26de, role_id=3d88c071954947f99ede3e31fb58494a)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:33.118537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ec660-4dff-48f2-a013-07a130a4f7d2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:33.139433 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 134/289] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:28:32 2022] PUT /identity/v3/projects/ae719f7141f34603811594a41497e87c/users/563026d078554a48a64feb283b6d26de/roles/3d88c071954947f99ede3e31fb58494a => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:33.205533 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdc90e-3a05-4b81-89b8-6e3e70ad0364 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:33.245948 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92485da8-cc18-4fc5-ab9b-8932c5a2e6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:33.246778 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92485da8-cc18-4fc5-ab9b-8932c5a2e6cf None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:33.247072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92485da8-cc18-4fc5-ab9b-8932c5a2e6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:33.247350 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92485da8-cc18-4fc5-ab9b-8932c5a2e6cf None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:33.248119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92485da8-cc18-4fc5-ab9b-8932c5a2e6cf None admin] RBAC: Adding query filter params (name=member) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:33.248432 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92485da8-cc18-4fc5-ab9b-8932c5a2e6cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:33.250610 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92485da8-cc18-4fc5-ab9b-8932c5a2e6cf None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:33.265543 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 156/290] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:33.266976 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ec660-4dff-48f2-a013-07a130a4f7d2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:33.416111 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ec660-4dff-48f2-a013-07a130a4f7d2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:33.435606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:33.470828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946daa37-de4a-4b50-8e1a-b42c38b91c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:33.472099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946daa37-de4a-4b50-8e1a-b42c38b91c51 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:33.472576 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946daa37-de4a-4b50-8e1a-b42c38b91c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:33.473060 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946daa37-de4a-4b50-8e1a-b42c38b91c51 None admin] PATH_INFO: `/v3/users/37cb64e3970448d7ac84496d1b59dd7d` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:33.474539 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946daa37-de4a-4b50-8e1a-b42c38b91c51 None admin] RBAC: Authorizing `identity:get_user(user_id=37cb64e3970448d7ac84496d1b59dd7d)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:33.476275 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946daa37-de4a-4b50-8e1a-b42c38b91c51 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:33.478614 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 135/291] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:33 2022] GET /identity/v3/users/37cb64e3970448d7ac84496d1b59dd7d => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:33.484528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92485da8-cc18-4fc5-ab9b-8932c5a2e6cf None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:33.584449 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:33.603954 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92485da8-cc18-4fc5-ab9b-8932c5a2e6cf None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:33.653416 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb208f9-ad13-4b86-a11d-ec73ca1ec9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:33.654117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb208f9-ad13-4b86-a11d-ec73ca1ec9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:33.654407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb208f9-ad13-4b86-a11d-ec73ca1ec9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:33.654689 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb208f9-ad13-4b86-a11d-ec73ca1ec9f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:33.734739 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb208f9-ad13-4b86-a11d-ec73ca1ec9f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:33.736299 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb208f9-ad13-4b86-a11d-ec73ca1ec9f4 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:33.845327 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 157/292] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:33.845989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:33.853272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946daa37-de4a-4b50-8e1a-b42c38b91c51 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:34.012990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946daa37-de4a-4b50-8e1a-b42c38b91c51 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:34.063588 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c3bac-4316-475a-9de1-1474e2c17921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:34.064472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c3bac-4316-475a-9de1-1474e2c17921 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:34.064759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c3bac-4316-475a-9de1-1474e2c17921 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:34.065039 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c3bac-4316-475a-9de1-1474e2c17921 None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:34.066307 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c3bac-4316-475a-9de1-1474e2c17921 None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:34.067785 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c3bac-4316-475a-9de1-1474e2c17921 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:34.069759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 136/293] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:33 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:34.075832 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb208f9-ad13-4b86-a11d-ec73ca1ec9f4 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:34.251752 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb208f9-ad13-4b86-a11d-ec73ca1ec9f4 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:34.293470 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8982d-31a0-4f5f-b3c8-c6f523e3ed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:34.294164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8982d-31a0-4f5f-b3c8-c6f523e3ed31 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:34.294455 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8982d-31a0-4f5f-b3c8-c6f523e3ed31 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:34.294736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8982d-31a0-4f5f-b3c8-c6f523e3ed31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:34.295598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8982d-31a0-4f5f-b3c8-c6f523e3ed31 None admin] RBAC: Adding query filter params (role.id=ed842da869724411a5b355cd12f1c5c8, scope.project.id=ae719f7141f34603811594a41497e87c, user.id=37cb64e3970448d7ac84496d1b59dd7d) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:34.295930 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8982d-31a0-4f5f-b3c8-c6f523e3ed31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:34.297384 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8982d-31a0-4f5f-b3c8-c6f523e3ed31 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:34.304114 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 158/294] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:34 2022] GET /identity/v3/role_assignments?user.id=37cb64e3970448d7ac84496d1b59dd7d&scope.project.id=ae719f7141f34603811594a41497e87c&role.id=ed842da869724411a5b355cd12f1c5c8 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:34.363157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e110f3-3e20-4984-be1f-58ada6038af2 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:34.363503 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e110f3-3e20-4984-be1f-58ada6038af2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:34.363717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e110f3-3e20-4984-be1f-58ada6038af2 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:34.365371 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 137/295] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:34.370886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123a28c-7870-4e02-8f77-1d346a4c0f51 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:34.371179 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123a28c-7870-4e02-8f77-1d346a4c0f51 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:34.371456 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123a28c-7870-4e02-8f77-1d346a4c0f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:34.395055 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5123a28c-7870-4e02-8f77-1d346a4c0f51 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:34.396280 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5123a28c-7870-4e02-8f77-1d346a4c0f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:34.451591 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:34.487885 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 159/296] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:34.525629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e110f3-3e20-4984-be1f-58ada6038af2 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:34.579855 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bfe58-70a4-43cc-981c-9d1020912937 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:34.580174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bfe58-70a4-43cc-981c-9d1020912937 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:34.580418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bfe58-70a4-43cc-981c-9d1020912937 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:34.582041 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 160/297] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:34.587614 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621f182-7075-41c0-bb3f-594e97194527 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:34.587907 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621f182-7075-41c0-bb3f-594e97194527 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:34.588177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621f182-7075-41c0-bb3f-594e97194527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:34.612628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-6621f182-7075-41c0-bb3f-594e97194527 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:34.614236 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6621f182-7075-41c0-bb3f-594e97194527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:34.614362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:34.687220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e110f3-3e20-4984-be1f-58ada6038af2 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:34.698469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 161/298] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:34.722458 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621f182-7075-41c0-bb3f-594e97194527 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:34.730751 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3e1cc-9be9-4aa5-9cd3-21defb9d85ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:34.731623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e1cc-9be9-4aa5-9cd3-21defb9d85ff None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:34.731952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e1cc-9be9-4aa5-9cd3-21defb9d85ff None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:34.732229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e1cc-9be9-4aa5-9cd3-21defb9d85ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:34.745234 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3e1cc-9be9-4aa5-9cd3-21defb9d85ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:34.754358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-24d3e1cc-9be9-4aa5-9cd3-21defb9d85ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:34.754358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:34.754358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:34.754358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:34.754358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:34.754358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:34.754358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:34.754358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:34.754358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:34.754358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:34.754358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:34.754358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:34.754358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:34.754358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:34.754358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:34.754358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:34.755257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:34.755257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:34.755257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:34.755257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:34.755257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:34.755257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:34.755257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:34.755257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:34.755257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:34.755257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:34.755257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:34.755257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:34.755257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:34.755257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:34.755257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:34.755257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:34.756308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:34.756308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:34.756308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:34.756308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:34.756308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:34.756308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:34.756308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:34.756308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:34.756308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:34.756308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:34.756308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:34.756308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:34.756308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 138/299] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:34.761270 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3e1cc-9be9-4aa5-9cd3-21defb9d85ff None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:34.875086 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621f182-7075-41c0-bb3f-594e97194527 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:34.887051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:34.919428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3e1cc-9be9-4aa5-9cd3-21defb9d85ff None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:34.928173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:34.934726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2e904-b071-43f2-aa32-a5ae033f48b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:34.936016 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2e904-b071-43f2-aa32-a5ae033f48b7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:34.936476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2e904-b071-43f2-aa32-a5ae033f48b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:34.937176 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2e904-b071-43f2-aa32-a5ae033f48b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:34.938702 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2e904-b071-43f2-aa32-a5ae033f48b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:34.940180 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2e904-b071-43f2-aa32-a5ae033f48b7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:34.946329 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 162/300] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:28:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:34.949256 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2e904-b071-43f2-aa32-a5ae033f48b7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:34.968903 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16700dc-2b10-4f52-b56c-6383d14f5996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:34.969605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16700dc-2b10-4f52-b56c-6383d14f5996 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:34.969882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16700dc-2b10-4f52-b56c-6383d14f5996 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:34.970187 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16700dc-2b10-4f52-b56c-6383d14f5996 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:34.970942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16700dc-2b10-4f52-b56c-6383d14f5996 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:34.971243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16700dc-2b10-4f52-b56c-6383d14f5996 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:34.972615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16700dc-2b10-4f52-b56c-6383d14f5996 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:34.985315 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 139/301] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:34.991128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16700dc-2b10-4f52-b56c-6383d14f5996 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:35.070406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2e904-b071-43f2-aa32-a5ae033f48b7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:35.100680 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16700dc-2b10-4f52-b56c-6383d14f5996 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:35.106358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca3c83-8fae-443e-849d-d4b8e21871d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:35.107023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca3c83-8fae-443e-849d-d4b8e21871d8 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:35.107389 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca3c83-8fae-443e-849d-d4b8e21871d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:35.107839 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca3c83-8fae-443e-849d-d4b8e21871d8 None admin] PATH_INFO: `/v3/groups` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:35.109014 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca3c83-8fae-443e-849d-d4b8e21871d8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:35.110608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca3c83-8fae-443e-849d-d4b8e21871d8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:35.125899 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 163/302] 10.222.0.52 () {60 vars in 1134 bytes} [Tue Jul 12 13:28:34 2022] POST /identity/v3/groups => generated 229 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:28:35.135626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27111d6f-df10-4041-8258-636af860d40a None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:35.135935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27111d6f-df10-4041-8258-636af860d40a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:35.136190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27111d6f-df10-4041-8258-636af860d40a None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:35.137137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85cca59-c794-41df-8513-a31bc4aa2efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:35.142116 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 164/303] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:35.142301 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cca59-c794-41df-8513-a31bc4aa2efb None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:35.142301 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cca59-c794-41df-8513-a31bc4aa2efb None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:35.142301 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cca59-c794-41df-8513-a31bc4aa2efb None admin] PATH_INFO: `/v3/users/7d9b0606748b4db2a65872a64d57f98c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:35.143472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556f7f1-08a3-4d96-894b-5e6bc9bbf30c None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:35.143755 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556f7f1-08a3-4d96-894b-5e6bc9bbf30c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:35.144029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556f7f1-08a3-4d96-894b-5e6bc9bbf30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:35.164800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85cca59-c794-41df-8513-a31bc4aa2efb None admin] RBAC: Authorizing `identity:get_user(user_id=7d9b0606748b4db2a65872a64d57f98c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:35.167109 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85cca59-c794-41df-8513-a31bc4aa2efb None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:35.169377 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 140/304] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:34 2022] GET /identity/v3/users/7d9b0606748b4db2a65872a64d57f98c => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:35.173205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b556f7f1-08a3-4d96-894b-5e6bc9bbf30c None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:35.174789 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b556f7f1-08a3-4d96-894b-5e6bc9bbf30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:35.176105 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85cca59-c794-41df-8513-a31bc4aa2efb None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:35.207947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:35.211741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:35.261983 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 165/305] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:35.285353 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556f7f1-08a3-4d96-894b-5e6bc9bbf30c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:35.309694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85cca59-c794-41df-8513-a31bc4aa2efb None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:35.368214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbe78f-d7a7-40f8-b455-b65438396758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:35.369049 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbe78f-d7a7-40f8-b455-b65438396758 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:35.369334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbe78f-d7a7-40f8-b455-b65438396758 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:35.369607 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbe78f-d7a7-40f8-b455-b65438396758 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:35.371270 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbe78f-d7a7-40f8-b455-b65438396758 None admin] RBAC: Adding query filter params (role.id=3d88c071954947f99ede3e31fb58494a, user.id=7d9b0606748b4db2a65872a64d57f98c, scope.system=all) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:35.371589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbe78f-d7a7-40f8-b455-b65438396758 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:35.373387 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbe78f-d7a7-40f8-b455-b65438396758 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:35.380727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 141/306] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Jul 12 13:28:35 2022] GET /identity/v3/role_assignments?user.id=7d9b0606748b4db2a65872a64d57f98c&scope.system=all&role.id=3d88c071954947f99ede3e31fb58494a => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:35.417951 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556f7f1-08a3-4d96-894b-5e6bc9bbf30c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:35.458221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00241e93-f57b-4cd6-98d6-66d3fb4bc6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:35.459036 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00241e93-f57b-4cd6-98d6-66d3fb4bc6fd None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:35.459316 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00241e93-f57b-4cd6-98d6-66d3fb4bc6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:35.459588 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00241e93-f57b-4cd6-98d6-66d3fb4bc6fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:35.472697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00241e93-f57b-4cd6-98d6-66d3fb4bc6fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:35.480709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-00241e93-f57b-4cd6-98d6-66d3fb4bc6fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:35.480709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:35.480709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:35.480709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:35.480709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:35.480709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:35.480709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:35.480709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:35.480709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:35.480709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:35.480709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:35.480709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:35.480709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:35.480709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:35.480709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:35.480709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:35.481673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:35.481673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:35.481673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:35.481673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:35.481673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:35.481673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:35.481673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:35.481673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:35.481673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:35.481673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:35.481673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:35.481673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:35.481673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:35.481673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:35.481673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:35.481673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:35.482479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:35.482479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:35.482479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:35.482479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:35.482479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:35.482479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:35.482479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:35.482479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:35.482479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:35.482479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:35.482479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:35.482479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:35.482479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 166/307] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:28:35.491160 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbe78f-d7a7-40f8-b455-b65438396758 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:35.593825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:35.604963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbe78f-d7a7-40f8-b455-b65438396758 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:35.641498 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06e8b6-071e-40e2-8533-c437f6cf4561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:35.642121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06e8b6-071e-40e2-8533-c437f6cf4561 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:35.642408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06e8b6-071e-40e2-8533-c437f6cf4561 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:35.642685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06e8b6-071e-40e2-8533-c437f6cf4561 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:35.643435 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06e8b6-071e-40e2-8533-c437f6cf4561 None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:35.643790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06e8b6-071e-40e2-8533-c437f6cf4561 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:35.645177 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06e8b6-071e-40e2-8533-c437f6cf4561 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:35.654075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 142/308] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:35.660599 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00241e93-f57b-4cd6-98d6-66d3fb4bc6fd None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:35.691950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:35.774261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00241e93-f57b-4cd6-98d6-66d3fb4bc6fd None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:35.808887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c0c9b-65e7-424f-8867-4a13b42f9ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:35.809764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c0c9b-65e7-424f-8867-4a13b42f9ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:35.810069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c0c9b-65e7-424f-8867-4a13b42f9ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:35.810346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c0c9b-65e7-424f-8867-4a13b42f9ec5 None admin] PATH_INFO: `/v3/users/1651993e33964848973f033abfe8003b` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:35.812099 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c0c9b-65e7-424f-8867-4a13b42f9ec5 None admin] RBAC: Authorizing `identity:get_user(user_id=1651993e33964848973f033abfe8003b)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:35.813542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c0c9b-65e7-424f-8867-4a13b42f9ec5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:35.815221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 167/309] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:35 2022] GET /identity/v3/users/1651993e33964848973f033abfe8003b => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:35.820784 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06e8b6-071e-40e2-8533-c437f6cf4561 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:35.847896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e7317-0d62-4d46-a73d-816c7093de9a None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:35.848364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e7317-0d62-4d46-a73d-816c7093de9a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:35.848472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e7317-0d62-4d46-a73d-816c7093de9a None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:35.850633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 168/310] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:35.859314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a1d28-8192-4c38-8bed-83423853961b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:35.859686 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a1d28-8192-4c38-8bed-83423853961b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:35.860029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a1d28-8192-4c38-8bed-83423853961b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:35.881820 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-fb9a1d28-8192-4c38-8bed-83423853961b None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:35.883134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fb9a1d28-8192-4c38-8bed-83423853961b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:35.939235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06e8b6-071e-40e2-8533-c437f6cf4561 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:35.963632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 169/311] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:35.975175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34aac1-f7bf-470f-839b-4b6f2e27a402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:35.976112 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34aac1-f7bf-470f-839b-4b6f2e27a402 None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:35.976462 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34aac1-f7bf-470f-839b-4b6f2e27a402 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:35.976803 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34aac1-f7bf-470f-839b-4b6f2e27a402 None admin] PATH_INFO: `/v3/system/users/1651993e33964848973f033abfe8003b/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:35.978382 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34aac1-f7bf-470f-839b-4b6f2e27a402 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1651993e33964848973f033abfe8003b, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:35.979831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34aac1-f7bf-470f-839b-4b6f2e27a402 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:35.988573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a1d28-8192-4c38-8bed-83423853961b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:35.991428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 143/312] 10.222.0.52 () {58 vars in 1334 bytes} [Tue Jul 12 13:28:35 2022] PUT /identity/v3/system/users/1651993e33964848973f033abfe8003b/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:36.019108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:36.106638 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a1d28-8192-4c38-8bed-83423853961b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:36.146301 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12dfabe-357c-4df6-b42a-0288541510f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:36.147124 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dfabe-357c-4df6-b42a-0288541510f2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:36.147409 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dfabe-357c-4df6-b42a-0288541510f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:36.147688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dfabe-357c-4df6-b42a-0288541510f2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:36.159597 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12dfabe-357c-4df6-b42a-0288541510f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:36.167515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-f12dfabe-357c-4df6-b42a-0288541510f2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:36.167515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:36.167515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:36.167515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:36.167515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:36.167515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:36.167515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:36.167515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:36.167515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:36.167515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:36.167515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:36.167515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:36.167515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:36.167515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:36.167515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:36.167515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:36.168373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:36.168373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:36.168373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:36.168373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:36.168373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:36.168373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:36.168373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:36.168373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:36.168373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:36.168373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:36.168373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:36.168373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:36.168373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:36.168373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:36.168373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:36.168373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:36.169134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:36.169134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:36.169134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:36.169134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:36.169134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:36.169134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:36.169134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:36.169134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:36.169134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:36.169134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:36.169134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:36.169134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:36.169134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 170/313] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:28:36.175295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34aac1-f7bf-470f-839b-4b6f2e27a402 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:36.197671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:36.284330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34aac1-f7bf-470f-839b-4b6f2e27a402 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:36.319858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca829300-8c84-46a7-af05-104e41ca7f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:36.320833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca829300-8c84-46a7-af05-104e41ca7f09 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:36.321327 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca829300-8c84-46a7-af05-104e41ca7f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:36.321779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca829300-8c84-46a7-af05-104e41ca7f09 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:36.322732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca829300-8c84-46a7-af05-104e41ca7f09 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:36.323276 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca829300-8c84-46a7-af05-104e41ca7f09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:36.324876 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca829300-8c84-46a7-af05-104e41ca7f09 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:36.334054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 144/314] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:36.340005 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12dfabe-357c-4df6-b42a-0288541510f2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:36.430641 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:36.450278 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12dfabe-357c-4df6-b42a-0288541510f2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:36.487211 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628779d2-397e-4d3a-98fa-31ef48809f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:36.488096 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628779d2-397e-4d3a-98fa-31ef48809f89 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:36.488375 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628779d2-397e-4d3a-98fa-31ef48809f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:36.488648 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628779d2-397e-4d3a-98fa-31ef48809f89 None admin] PATH_INFO: `/v3/users/563026d078554a48a64feb283b6d26de` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:36.489893 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628779d2-397e-4d3a-98fa-31ef48809f89 None admin] RBAC: Authorizing `identity:get_user(user_id=563026d078554a48a64feb283b6d26de)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:36.491385 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628779d2-397e-4d3a-98fa-31ef48809f89 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:36.493332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 171/315] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:36 2022] GET /identity/v3/users/563026d078554a48a64feb283b6d26de => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:36.499198 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca829300-8c84-46a7-af05-104e41ca7f09 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:36.585751 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:36.608715 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca829300-8c84-46a7-af05-104e41ca7f09 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:36.642767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca57ba-03bc-42df-b97b-1d050f7a3adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:36.643419 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca57ba-03bc-42df-b97b-1d050f7a3adb None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:36.643697 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca57ba-03bc-42df-b97b-1d050f7a3adb None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:36.643967 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca57ba-03bc-42df-b97b-1d050f7a3adb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:36.715739 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca57ba-03bc-42df-b97b-1d050f7a3adb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:36.717183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca57ba-03bc-42df-b97b-1d050f7a3adb None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:36.806992 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:36.818915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 145/316] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:36.824587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628779d2-397e-4d3a-98fa-31ef48809f89 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:36.926511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628779d2-397e-4d3a-98fa-31ef48809f89 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:36.960530 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418f575-d1dc-4f8f-b376-51c4cf4ec72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:36.961398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418f575-d1dc-4f8f-b376-51c4cf4ec72f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:36.961728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418f575-d1dc-4f8f-b376-51c4cf4ec72f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:36.962024 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418f575-d1dc-4f8f-b376-51c4cf4ec72f None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:36.963212 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a418f575-d1dc-4f8f-b376-51c4cf4ec72f None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:36.964658 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a418f575-d1dc-4f8f-b376-51c4cf4ec72f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:36.966886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 172/317] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:36 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:36.975041 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca57ba-03bc-42df-b97b-1d050f7a3adb None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:37.080432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca57ba-03bc-42df-b97b-1d050f7a3adb None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:37.115376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b9ecf-4c09-4ed7-bc1e-34680672c31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:37.116076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b9ecf-4c09-4ed7-bc1e-34680672c31d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.116358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b9ecf-4c09-4ed7-bc1e-34680672c31d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.116629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b9ecf-4c09-4ed7-bc1e-34680672c31d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.117451 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b9ecf-4c09-4ed7-bc1e-34680672c31d None admin] RBAC: Adding query filter params (role.id=3d88c071954947f99ede3e31fb58494a, scope.project.id=ae719f7141f34603811594a41497e87c, user.id=563026d078554a48a64feb283b6d26de) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:37.117759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b9ecf-4c09-4ed7-bc1e-34680672c31d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:37.119222 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b9ecf-4c09-4ed7-bc1e-34680672c31d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:37.125325 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 146/318] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:28:36 2022] GET /identity/v3/role_assignments?user.id=563026d078554a48a64feb283b6d26de&scope.project.id=ae719f7141f34603811594a41497e87c&role.id=3d88c071954947f99ede3e31fb58494a => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:37.314634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01d068-c8f7-4b59-a7b6-47bceb6b33c0 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.315289 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01d068-c8f7-4b59-a7b6-47bceb6b33c0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.315576 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01d068-c8f7-4b59-a7b6-47bceb6b33c0 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.317084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 173/319] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:37.322537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d606c2-ec08-4228-abdc-f2555689249d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.323068 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d606c2-ec08-4228-abdc-f2555689249d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.323548 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d606c2-ec08-4228-abdc-f2555689249d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.341207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-26d606c2-ec08-4228-abdc-f2555689249d None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:37.342830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-26d606c2-ec08-4228-abdc-f2555689249d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:37.381011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:37.413521 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 147/320] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:37.435732 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01d068-c8f7-4b59-a7b6-47bceb6b33c0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:37.456951 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:37.533822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88e2db-5126-4939-aaf7-20d369091b74 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.534157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88e2db-5126-4939-aaf7-20d369091b74 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.534718 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88e2db-5126-4939-aaf7-20d369091b74 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.536316 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 148/321] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:28:37.541066 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ecb7c-826c-4289-8751-4d24df1ee721 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.541363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ecb7c-826c-4289-8751-4d24df1ee721 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.541643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ecb7c-826c-4289-8751-4d24df1ee721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.547585 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01d068-c8f7-4b59-a7b6-47bceb6b33c0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:37.560097 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a09ecb7c-826c-4289-8751-4d24df1ee721 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:37.561835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a09ecb7c-826c-4289-8751-4d24df1ee721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:37.587602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afea161-2e30-4f6c-a1ed-60e63fbf4a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:37.588475 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afea161-2e30-4f6c-a1ed-60e63fbf4a3a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.588760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afea161-2e30-4f6c-a1ed-60e63fbf4a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.589040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afea161-2e30-4f6c-a1ed-60e63fbf4a3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.590251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afea161-2e30-4f6c-a1ed-60e63fbf4a3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:37.591693 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afea161-2e30-4f6c-a1ed-60e63fbf4a3a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:37.593618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 174/322] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:28:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:37.598608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afea161-2e30-4f6c-a1ed-60e63fbf4a3a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:37.635851 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 149/323] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:37.657678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ecb7c-826c-4289-8751-4d24df1ee721 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:37.694902 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:37.713529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afea161-2e30-4f6c-a1ed-60e63fbf4a3a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:37.751411 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5655f9-604d-430f-aeda-c84246901b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:37.752086 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5655f9-604d-430f-aeda-c84246901b08 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.752367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5655f9-604d-430f-aeda-c84246901b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.752903 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5655f9-604d-430f-aeda-c84246901b08 None admin] PATH_INFO: `/v3/groups` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.753600 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5655f9-604d-430f-aeda-c84246901b08 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:37.755083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5655f9-604d-430f-aeda-c84246901b08 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:37.769550 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ecb7c-826c-4289-8751-4d24df1ee721 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:37.770885 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 175/324] 10.222.0.52 () {60 vars in 1134 bytes} [Tue Jul 12 13:28:37 2022] POST /identity/v3/groups => generated 226 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:28:37.771306 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:37.812814 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e6648-733e-4ed3-91b0-63e1f6b9aa3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:37.813630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e6648-733e-4ed3-91b0-63e1f6b9aa3d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.813916 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e6648-733e-4ed3-91b0-63e1f6b9aa3d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.814227 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e6648-733e-4ed3-91b0-63e1f6b9aa3d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.827473 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e6648-733e-4ed3-91b0-63e1f6b9aa3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:37.835894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-a38e6648-733e-4ed3-91b0-63e1f6b9aa3d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:37.835894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:37.835894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:37.835894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:37.835894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:37.835894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:37.835894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:37.835894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:37.835894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:37.835894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:37.835894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:37.835894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:37.835894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:37.835894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:37.835894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:37.835894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:37.836937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:37.836937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:37.836937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:37.836937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:37.836937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:37.836937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:37.836937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:37.836937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:37.836937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:37.836937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:37.836937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:37.836937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:37.836937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:37.836937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:37.836937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:37.836937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:37.843170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:37.843170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:37.843170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:37.843170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:37.843170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:37.843170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:37.843170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:37.843170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:37.843170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:37.843170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:37.843170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:37.843170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:37.843170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 150/325] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:28:37.845582 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5655f9-604d-430f-aeda-c84246901b08 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:37.957374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5655f9-604d-430f-aeda-c84246901b08 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:37.992539 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b55dc1-8e9e-43dc-a017-7cf5aaef560f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:37.993204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b55dc1-8e9e-43dc-a017-7cf5aaef560f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.993484 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b55dc1-8e9e-43dc-a017-7cf5aaef560f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.993757 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b55dc1-8e9e-43dc-a017-7cf5aaef560f None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.994531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b55dc1-8e9e-43dc-a017-7cf5aaef560f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:37.994978 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b55dc1-8e9e-43dc-a017-7cf5aaef560f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:37.996242 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b55dc1-8e9e-43dc-a017-7cf5aaef560f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:38.006564 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 176/326] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:38.012157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e6648-733e-4ed3-91b0-63e1f6b9aa3d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:38.117804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e6648-733e-4ed3-91b0-63e1f6b9aa3d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:38.148989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90bb3e-2ff7-4e0c-b92d-28c43146e55a None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:38.149320 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90bb3e-2ff7-4e0c-b92d-28c43146e55a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:38.149538 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90bb3e-2ff7-4e0c-b92d-28c43146e55a None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.150954 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 177/327] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:28:38.156999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a648b9-ff48-48c8-9098-d3aa0fb30b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:38.157876 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a648b9-ff48-48c8-9098-d3aa0fb30b99 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:38.158183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a648b9-ff48-48c8-9098-d3aa0fb30b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:38.158456 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a648b9-ff48-48c8-9098-d3aa0fb30b99 None admin] PATH_INFO: `/v3/users/7d9b0606748b4db2a65872a64d57f98c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.159166 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1ad6d-0f55-42e6-b3dc-0d56a6e6b2ef None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:38.159656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1ad6d-0f55-42e6-b3dc-0d56a6e6b2ef None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:38.160126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1ad6d-0f55-42e6-b3dc-0d56a6e6b2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.160709 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a648b9-ff48-48c8-9098-d3aa0fb30b99 None admin] RBAC: Authorizing `identity:get_user(user_id=7d9b0606748b4db2a65872a64d57f98c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:38.162362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a648b9-ff48-48c8-9098-d3aa0fb30b99 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:38.164182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 151/328] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:38 2022] GET /identity/v3/users/7d9b0606748b4db2a65872a64d57f98c => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:38.169390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a648b9-ff48-48c8-9098-d3aa0fb30b99 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:38.185714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-72e1ad6d-0f55-42e6-b3dc-0d56a6e6b2ef None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:38.187060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-72e1ad6d-0f55-42e6-b3dc-0d56a6e6b2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:38.189529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:38.215676 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:38.276727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 178/329] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:38.291952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a648b9-ff48-48c8-9098-d3aa0fb30b99 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:38.301649 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1ad6d-0f55-42e6-b3dc-0d56a6e6b2ef None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:38.328178 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179d4bb-ad80-4b11-b443-3c14fa07fe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:38.329052 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179d4bb-ad80-4b11-b443-3c14fa07fe07 None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:38.329383 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179d4bb-ad80-4b11-b443-3c14fa07fe07 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:38.329666 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179d4bb-ad80-4b11-b443-3c14fa07fe07 None admin] PATH_INFO: `/v3/system/users/7d9b0606748b4db2a65872a64d57f98c/roles/3d88c071954947f99ede3e31fb58494a` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.331300 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179d4bb-ad80-4b11-b443-3c14fa07fe07 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7d9b0606748b4db2a65872a64d57f98c, role_id=3d88c071954947f99ede3e31fb58494a)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:38.332696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179d4bb-ad80-4b11-b443-3c14fa07fe07 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:38.341917 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 152/330] 10.222.0.52 () {58 vars in 1334 bytes} [Tue Jul 12 13:28:38 2022] PUT /identity/v3/system/users/7d9b0606748b4db2a65872a64d57f98c/roles/3d88c071954947f99ede3e31fb58494a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:38.421458 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1ad6d-0f55-42e6-b3dc-0d56a6e6b2ef None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:38.461371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428a828-aef5-4377-9cfb-db2e3b0d135d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:38.462287 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428a828-aef5-4377-9cfb-db2e3b0d135d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:38.462578 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428a828-aef5-4377-9cfb-db2e3b0d135d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:38.462859 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428a828-aef5-4377-9cfb-db2e3b0d135d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.472044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:38.476398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428a828-aef5-4377-9cfb-db2e3b0d135d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:38.486447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-f428a828-aef5-4377-9cfb-db2e3b0d135d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:38.486447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:38.486447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:38.486447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:38.486447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:38.486447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:38.486447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:38.486447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:38.486447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:38.486447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:38.486447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:38.486447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:38.486447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:38.486447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:38.486447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:38.486447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:38.487390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:38.487390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:38.487390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:38.487390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:38.487390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:38.487390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:38.487390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:38.487390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:38.487390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:38.487390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:38.487390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:38.487390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:38.487390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:38.487390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:38.487390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:38.487390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:38.488204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:38.488204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:38.488204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:38.488204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:38.488204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:38.488204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:38.488204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:38.488204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:38.488204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:38.488204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:38.488204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:38.488204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:38.488204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 179/331] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:28:38.495257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179d4bb-ad80-4b11-b443-3c14fa07fe07 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:38.599890 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:38.607909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179d4bb-ad80-4b11-b443-3c14fa07fe07 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:38.655981 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01382a-ef81-4440-b07a-5992f5318d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:38.656675 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01382a-ef81-4440-b07a-5992f5318d5e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:38.656961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01382a-ef81-4440-b07a-5992f5318d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:38.657240 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01382a-ef81-4440-b07a-5992f5318d5e None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.658106 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c01382a-ef81-4440-b07a-5992f5318d5e None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:38.658422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c01382a-ef81-4440-b07a-5992f5318d5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:38.659804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c01382a-ef81-4440-b07a-5992f5318d5e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:38.669551 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 153/332] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:38.675332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428a828-aef5-4377-9cfb-db2e3b0d135d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:38.787007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428a828-aef5-4377-9cfb-db2e3b0d135d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:38.832446 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f32d70-6165-4e85-9a17-c75f8eb07b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:38.833333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f32d70-6165-4e85-9a17-c75f8eb07b80 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:38.833615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f32d70-6165-4e85-9a17-c75f8eb07b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:38.833900 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f32d70-6165-4e85-9a17-c75f8eb07b80 None admin] PATH_INFO: `/v3/users/1651993e33964848973f033abfe8003b` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.836102 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f32d70-6165-4e85-9a17-c75f8eb07b80 None admin] RBAC: Authorizing `identity:get_user(user_id=1651993e33964848973f033abfe8003b)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:38.837590 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f32d70-6165-4e85-9a17-c75f8eb07b80 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:38.839602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 180/333] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:38 2022] GET /identity/v3/users/1651993e33964848973f033abfe8003b => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:38.849883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01382a-ef81-4440-b07a-5992f5318d5e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:39.005505 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01382a-ef81-4440-b07a-5992f5318d5e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:39.051707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9aacf-264a-49ad-89ed-105128ccb31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:39.052347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9aacf-264a-49ad-89ed-105128ccb31e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:39.052640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9aacf-264a-49ad-89ed-105128ccb31e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:39.052919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9aacf-264a-49ad-89ed-105128ccb31e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:39.053755 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9aacf-264a-49ad-89ed-105128ccb31e None admin] RBAC: Adding query filter params (role.id=ed842da869724411a5b355cd12f1c5c8, user.id=1651993e33964848973f033abfe8003b, scope.system=all) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:39.054097 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9aacf-264a-49ad-89ed-105128ccb31e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:39.055549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9aacf-264a-49ad-89ed-105128ccb31e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:39.073676 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 154/334] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Jul 12 13:28:38 2022] GET /identity/v3/role_assignments?user.id=1651993e33964848973f033abfe8003b&scope.system=all&role.id=ed842da869724411a5b355cd12f1c5c8 => generated 510 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:40.322663 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142af77-ed80-4d90-825c-311fc4aa518c None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:40.323182 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142af77-ed80-4d90-825c-311fc4aa518c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:40.323363 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142af77-ed80-4d90-825c-311fc4aa518c None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:40.324840 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 181/335] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:40.330471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ac135-a549-49ec-85dc-672b8b67da1e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:40.330928 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ac135-a549-49ec-85dc-672b8b67da1e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:40.331114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ac135-a549-49ec-85dc-672b8b67da1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:40.355023 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-758ac135-a549-49ec-85dc-672b8b67da1e None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:40.356248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-758ac135-a549-49ec-85dc-672b8b67da1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:40.367227 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:40.440582 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 155/336] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:40.484313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142af77-ed80-4d90-825c-311fc4aa518c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:40.582427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:40.629889 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142af77-ed80-4d90-825c-311fc4aa518c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:40.676574 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cfc57-e2a0-4f42-b17d-84be260c9ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:40.677411 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cfc57-e2a0-4f42-b17d-84be260c9ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:40.677696 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cfc57-e2a0-4f42-b17d-84be260c9ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:40.677992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cfc57-e2a0-4f42-b17d-84be260c9ab8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:40.695905 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149cfc57-e2a0-4f42-b17d-84be260c9ab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:40.704845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-149cfc57-e2a0-4f42-b17d-84be260c9ab8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:40.704845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:40.704845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:40.704845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:40.704845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:40.704845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:40.704845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:40.704845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:40.704845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:40.704845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:40.704845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:40.704845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:40.704845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:40.704845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:40.704845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:40.704845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:40.705719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:40.705719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:40.705719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:40.705719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:40.705719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:40.705719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:40.705719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:40.705719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:40.705719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:40.705719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:40.705719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:40.705719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:40.705719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:40.705719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:40.705719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:40.705719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:40.706560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:40.706560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:40.706560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:40.706560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:40.706560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:40.706560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:40.706560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:40.706560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:40.706560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:40.706560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:40.706560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:40.706560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:40.710480 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 182/337] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:28:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:40.716400 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ac135-a549-49ec-85dc-672b8b67da1e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:40.852454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ac135-a549-49ec-85dc-672b8b67da1e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:40.893464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b01c89-14cd-4a2f-aa6e-91b6914ddd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:40.894170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b01c89-14cd-4a2f-aa6e-91b6914ddd9b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:40.894454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b01c89-14cd-4a2f-aa6e-91b6914ddd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:40.894729 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b01c89-14cd-4a2f-aa6e-91b6914ddd9b None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:40.895486 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b01c89-14cd-4a2f-aa6e-91b6914ddd9b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:40.895790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b01c89-14cd-4a2f-aa6e-91b6914ddd9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:40.897174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b01c89-14cd-4a2f-aa6e-91b6914ddd9b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:40.906851 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 156/338] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:28:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:40.909567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d77d9-adf5-43a4-a36a-6479778e07c5 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:40.909865 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d77d9-adf5-43a4-a36a-6479778e07c5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:40.910188 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d77d9-adf5-43a4-a36a-6479778e07c5 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:40.911892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 183/339] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:40.912205 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b01c89-14cd-4a2f-aa6e-91b6914ddd9b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:40.917332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d13ee6-43bb-4222-9f8b-13be26a465e1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:40.917706 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d13ee6-43bb-4222-9f8b-13be26a465e1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:40.918012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d13ee6-43bb-4222-9f8b-13be26a465e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:40.926318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:40.954346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a1d13ee6-43bb-4222-9f8b-13be26a465e1 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:40.955582 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a1d13ee6-43bb-4222-9f8b-13be26a465e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:41.064588 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b01c89-14cd-4a2f-aa6e-91b6914ddd9b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:41.089751 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 184/340] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:41.114707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d13ee6-43bb-4222-9f8b-13be26a465e1 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:41.126690 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e557f6-7de0-4bf5-b04d-64cd45bb3b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:41.128059 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e557f6-7de0-4bf5-b04d-64cd45bb3b1a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:41.128566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e557f6-7de0-4bf5-b04d-64cd45bb3b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:41.129065 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e557f6-7de0-4bf5-b04d-64cd45bb3b1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:41.182193 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:41.219308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e557f6-7de0-4bf5-b04d-64cd45bb3b1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:41.222116 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e557f6-7de0-4bf5-b04d-64cd45bb3b1a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:41.232261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:41.242842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d13ee6-43bb-4222-9f8b-13be26a465e1 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:41.283619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884106a5-0876-41ce-b506-f5c34dcb376a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:41.284432 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884106a5-0876-41ce-b506-f5c34dcb376a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:41.284725 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884106a5-0876-41ce-b506-f5c34dcb376a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:41.285001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884106a5-0876-41ce-b506-f5c34dcb376a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:41.298123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884106a5-0876-41ce-b506-f5c34dcb376a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:41.306351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-884106a5-0876-41ce-b506-f5c34dcb376a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:41.306351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:41.306351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:41.306351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:41.306351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:41.306351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:41.306351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:41.306351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:41.306351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:41.306351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:41.306351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:41.306351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:41.306351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:41.306351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:41.306351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:41.306351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:41.307245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:41.307245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:41.307245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:41.307245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:41.307245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:41.307245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:41.307245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:41.307245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:41.307245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:41.307245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:41.307245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:41.307245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:41.307245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:41.307245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:41.307245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:41.307245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:41.308427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:41.308427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:41.308427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:41.308427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:41.308427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:41.308427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:41.308427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:41.308427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:41.308427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:41.308427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:41.308427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:28:41.308427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:41.308427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 185/341] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:41.315162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026170ce-4f75-4dd3-b53c-a9d4f4a2d129 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:41.315421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026170ce-4f75-4dd3-b53c-a9d4f4a2d129 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:41.315645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026170ce-4f75-4dd3-b53c-a9d4f4a2d129 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:41.317047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 186/342] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:41.318204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964d991-980c-4e7a-a51e-aa77642a6618 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:41.318468 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964d991-980c-4e7a-a51e-aa77642a6618 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:41.318748 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964d991-980c-4e7a-a51e-aa77642a6618 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:41.323762 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 187/343] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:41.323762 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964d991-980c-4e7a-a51e-aa77642a6618 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:41.364257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 157/344] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 13:28:41.365855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22855a88-fc12-439a-ba86-28d5257f2457 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:41.366213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22855a88-fc12-439a-ba86-28d5257f2457 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:41.366589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22855a88-fc12-439a-ba86-28d5257f2457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:41.388123 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-22855a88-fc12-439a-ba86-28d5257f2457 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:41.389266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-22855a88-fc12-439a-ba86-28d5257f2457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:41.465483 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964d991-980c-4e7a-a51e-aa77642a6618 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:41.469057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 158/345] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:41.470484 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da412b65-50ed-4eba-89dc-0fbed86a7ac5 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:41.470930 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da412b65-50ed-4eba-89dc-0fbed86a7ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:41.471334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da412b65-50ed-4eba-89dc-0fbed86a7ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:41.489692 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:41.491597 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-da412b65-50ed-4eba-89dc-0fbed86a7ac5 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:41.493830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-da412b65-50ed-4eba-89dc-0fbed86a7ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:41.517104 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d470b69-59de-454b-a0ee-02c8876a241a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:41.518052 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d470b69-59de-454b-a0ee-02c8876a241a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:41.518484 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d470b69-59de-454b-a0ee-02c8876a241a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:41.519463 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d470b69-59de-454b-a0ee-02c8876a241a None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:41.520526 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d470b69-59de-454b-a0ee-02c8876a241a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:41.521115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d470b69-59de-454b-a0ee-02c8876a241a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:41.522978 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d470b69-59de-454b-a0ee-02c8876a241a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:41.529589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:41.536427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 188/346] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:41.537620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d470b69-59de-454b-a0ee-02c8876a241a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:41.592126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 159/347] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:41.592807 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da412b65-50ed-4eba-89dc-0fbed86a7ac5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:41.678873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d470b69-59de-454b-a0ee-02c8876a241a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:41.715154 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da412b65-50ed-4eba-89dc-0fbed86a7ac5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:41.727813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3f1ec-a7e2-4904-a716-579c9dc415d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:41.729009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3f1ec-a7e2-4904-a716-579c9dc415d9 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:41.729297 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3f1ec-a7e2-4904-a716-579c9dc415d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:41.729577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3f1ec-a7e2-4904-a716-579c9dc415d9 None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:41.731314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3f1ec-a7e2-4904-a716-579c9dc415d9 None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:41.734061 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3f1ec-a7e2-4904-a716-579c9dc415d9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:41.736229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 189/348] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:41 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:41.742896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3f1ec-a7e2-4904-a716-579c9dc415d9 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:41.766735 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78c42a-7873-40fc-b6b5-375a3b8d4041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:41.767993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78c42a-7873-40fc-b6b5-375a3b8d4041 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:41.768501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78c42a-7873-40fc-b6b5-375a3b8d4041 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:41.768974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78c42a-7873-40fc-b6b5-375a3b8d4041 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:41.781117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78c42a-7873-40fc-b6b5-375a3b8d4041 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:41.789546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-fd78c42a-7873-40fc-b6b5-375a3b8d4041 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:41.789546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:41.789546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:41.789546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:41.789546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:41.789546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:41.789546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:41.789546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:41.789546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:41.789546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:41.789546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:41.789546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:41.789546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:41.789546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:41.789546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:41.789546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:41.790589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:41.790589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:41.790589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:41.790589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:41.790589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:41.790589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:41.790589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:41.790589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:41.790589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:41.790589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:41.790589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:41.790589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:41.790589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:41.790589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:41.790589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:41.790589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:41.791408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:41.791408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:41.791408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:41.791408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:41.791408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:41.791408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:41.791408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:41.791408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:41.791408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:41.791408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:41.791408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:41.791408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:41.793274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:41.794073 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 160/349] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:41.795154 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78c42a-7873-40fc-b6b5-375a3b8d4041 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:41.826397 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:41.854137 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3f1ec-a7e2-4904-a716-579c9dc415d9 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:41.892275 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64d069-4681-4b94-ab2d-8956c3298af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:41.893095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64d069-4681-4b94-ab2d-8956c3298af5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:41.893385 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64d069-4681-4b94-ab2d-8956c3298af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:41.893657 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64d069-4681-4b94-ab2d-8956c3298af5 None admin] PATH_INFO: `/v3/users/7d9b0606748b4db2a65872a64d57f98c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:41.895112 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64d069-4681-4b94-ab2d-8956c3298af5 None admin] RBAC: Authorizing `identity:get_user(user_id=7d9b0606748b4db2a65872a64d57f98c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:41.896448 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64d069-4681-4b94-ab2d-8956c3298af5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:41.898101 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 190/350] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:28:41 2022] GET /identity/v3/users/7d9b0606748b4db2a65872a64d57f98c => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:41.898962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64d069-4681-4b94-ab2d-8956c3298af5 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:41.912412 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78c42a-7873-40fc-b6b5-375a3b8d4041 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:41.948446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506e29d-78db-4000-ab10-50e97e00a5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:41.949565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506e29d-78db-4000-ab10-50e97e00a5fd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:41.950081 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506e29d-78db-4000-ab10-50e97e00a5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:41.950558 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506e29d-78db-4000-ab10-50e97e00a5fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:41.961752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506e29d-78db-4000-ab10-50e97e00a5fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:41.970682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-3506e29d-78db-4000-ab10-50e97e00a5fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:41.970682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:41.970682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:41.970682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:41.970682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:41.970682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:41.970682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:41.970682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:41.970682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:41.970682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:41.970682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:41.970682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:41.970682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:41.970682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:41.970682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:41.970682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:41.971603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:41.971603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:41.971603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:41.971603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:41.971603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:41.971603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:41.971603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:41.971603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:41.971603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:41.971603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:41.971603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:41.971603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:41.971603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:41.971603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:41.971603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:41.971603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:41.972414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:41.972414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:41.972414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:41.972414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:41.972414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:41.972414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:41.972414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:41.972414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:41.972414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:41.972414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:41.972414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:41.972414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:41.974791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 161/351] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:41.975797 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506e29d-78db-4000-ab10-50e97e00a5fd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:42.020428 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64d069-4681-4b94-ab2d-8956c3298af5 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:42.035593 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:42.061517 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d31697-76a2-400c-a2c9-2b5652110530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:42.062456 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d31697-76a2-400c-a2c9-2b5652110530 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:42.062743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d31697-76a2-400c-a2c9-2b5652110530 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:42.063025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d31697-76a2-400c-a2c9-2b5652110530 None admin] PATH_INFO: `/v3/groups/9758043b2c3a4673b089931b98c328e4` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:42.069435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d31697-76a2-400c-a2c9-2b5652110530 None admin] RBAC: Authorizing `identity:get_group(group_id=9758043b2c3a4673b089931b98c328e4)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:42.070274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:42.071047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d31697-76a2-400c-a2c9-2b5652110530 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:42.072724 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 191/352] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jul 12 13:28:41 2022] GET /identity/v3/groups/9758043b2c3a4673b089931b98c328e4 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:42.073642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d31697-76a2-400c-a2c9-2b5652110530 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:42.093295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506e29d-78db-4000-ab10-50e97e00a5fd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:42.133157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb46dfb-717e-4bc3-8a84-3dbafb28457e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:42.134211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb46dfb-717e-4bc3-8a84-3dbafb28457e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:42.134699 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb46dfb-717e-4bc3-8a84-3dbafb28457e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:42.135211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb46dfb-717e-4bc3-8a84-3dbafb28457e None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:42.136184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb46dfb-717e-4bc3-8a84-3dbafb28457e None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:42.136706 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb46dfb-717e-4bc3-8a84-3dbafb28457e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:42.138343 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb46dfb-717e-4bc3-8a84-3dbafb28457e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:42.147984 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 162/353] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:42.149180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb46dfb-717e-4bc3-8a84-3dbafb28457e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:42.182174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d31697-76a2-400c-a2c9-2b5652110530 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:42.220462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f46743-c7dd-4001-9632-88d5d7bcc964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:42.221052 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f46743-c7dd-4001-9632-88d5d7bcc964 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:42.221342 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f46743-c7dd-4001-9632-88d5d7bcc964 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:42.221621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f46743-c7dd-4001-9632-88d5d7bcc964 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:42.222498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f46743-c7dd-4001-9632-88d5d7bcc964 None admin] RBAC: Adding query filter params (role.id=3d88c071954947f99ede3e31fb58494a, user.id=7d9b0606748b4db2a65872a64d57f98c, scope.system=all) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:42.222815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f46743-c7dd-4001-9632-88d5d7bcc964 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:42.224282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f46743-c7dd-4001-9632-88d5d7bcc964 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:42.238769 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 192/354] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Jul 12 13:28:42 2022] GET /identity/v3/role_assignments?user.id=7d9b0606748b4db2a65872a64d57f98c&scope.system=all&role.id=3d88c071954947f99ede3e31fb58494a => generated 510 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:42.239709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f46743-c7dd-4001-9632-88d5d7bcc964 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:42.271896 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb46dfb-717e-4bc3-8a84-3dbafb28457e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:42.284152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:42.316921 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72125b08-898c-443e-be29-183d2e707e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:42.317612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72125b08-898c-443e-be29-183d2e707e87 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:42.317897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72125b08-898c-443e-be29-183d2e707e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:42.318203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72125b08-898c-443e-be29-183d2e707e87 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:42.318987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72125b08-898c-443e-be29-183d2e707e87 None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:42.319299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72125b08-898c-443e-be29-183d2e707e87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:42.324233 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72125b08-898c-443e-be29-183d2e707e87 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:42.342931 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 163/355] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:42.343806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72125b08-898c-443e-be29-183d2e707e87 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:42.373394 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:42.421093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f46743-c7dd-4001-9632-88d5d7bcc964 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:42.464450 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818d999-e423-4d8a-bc51-83704ff54743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:42.466516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818d999-e423-4d8a-bc51-83704ff54743 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:42.466516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818d999-e423-4d8a-bc51-83704ff54743 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:42.466516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818d999-e423-4d8a-bc51-83704ff54743 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:42.466516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818d999-e423-4d8a-bc51-83704ff54743 None admin] RBAC: Adding query filter params (group.id=9758043b2c3a4673b089931b98c328e4, role.id=dbe30950eded4c9ab2bcc0f7545b9bae, scope.project.id=abaed0828ac34adb801becefa86d65b3) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:42.466917 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818d999-e423-4d8a-bc51-83704ff54743 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:42.469322 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818d999-e423-4d8a-bc51-83704ff54743 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:42.475843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 193/356] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jul 12 13:28:42 2022] GET /identity/v3/role_assignments?group.id=9758043b2c3a4673b089931b98c328e4&scope.project.id=abaed0828ac34adb801becefa86d65b3&role.id=dbe30950eded4c9ab2bcc0f7545b9bae => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:42.476699 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818d999-e423-4d8a-bc51-83704ff54743 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:42.516217 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72125b08-898c-443e-be29-183d2e707e87 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:42.556941 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04012bd-c7ea-4175-9ed1-6f16fa001202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:42.557602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04012bd-c7ea-4175-9ed1-6f16fa001202 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:42.557893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04012bd-c7ea-4175-9ed1-6f16fa001202 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:42.558188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04012bd-c7ea-4175-9ed1-6f16fa001202 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:42.623897 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:42.627847 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818d999-e423-4d8a-bc51-83704ff54743 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:42.667888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04012bd-c7ea-4175-9ed1-6f16fa001202 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:42.670078 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04012bd-c7ea-4175-9ed1-6f16fa001202 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:42.681231 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76428f0e-3ba0-4ab6-9937-d1f6bdb0ffb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:42.681902 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76428f0e-3ba0-4ab6-9937-d1f6bdb0ffb7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:42.682335 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76428f0e-3ba0-4ab6-9937-d1f6bdb0ffb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:42.682479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76428f0e-3ba0-4ab6-9937-d1f6bdb0ffb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:42.691219 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:42.772403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76428f0e-3ba0-4ab6-9937-d1f6bdb0ffb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:42.773840 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76428f0e-3ba0-4ab6-9937-d1f6bdb0ffb7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:42.798410 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 164/357] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:42.800007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b71c3c-34a7-49b8-821f-3b73c196ed3c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:42.800447 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b71c3c-34a7-49b8-821f-3b73c196ed3c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:42.801033 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b71c3c-34a7-49b8-821f-3b73c196ed3c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:42.803883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 165/358] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:42.808966 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b71c3c-34a7-49b8-821f-3b73c196ed3c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:42.913721 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:42.918465 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 194/359] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:42.919466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4be8f-d7e3-4e26-addf-64566e5a228d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:42.919791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4be8f-d7e3-4e26-addf-64566e5a228d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:42.920150 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4be8f-d7e3-4e26-addf-64566e5a228d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:42.941603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-33d4be8f-d7e3-4e26-addf-64566e5a228d None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:42.942867 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-33d4be8f-d7e3-4e26-addf-64566e5a228d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:42.956633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b71c3c-34a7-49b8-821f-3b73c196ed3c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:42.972048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:43.004571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942b8ea-e33a-403c-9766-9b306d36169d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:43.005739 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942b8ea-e33a-403c-9766-9b306d36169d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:43.006188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942b8ea-e33a-403c-9766-9b306d36169d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:43.006550 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942b8ea-e33a-403c-9766-9b306d36169d None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:43.008630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942b8ea-e33a-403c-9766-9b306d36169d None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:43.009298 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942b8ea-e33a-403c-9766-9b306d36169d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:43.011535 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 166/360] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:42 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:43.012603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efdef2-8865-49eb-87e7-3a3036e7ca2c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:43.012910 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efdef2-8865-49eb-87e7-3a3036e7ca2c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:43.013163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efdef2-8865-49eb-87e7-3a3036e7ca2c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:43.015446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 167/361] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:43.016209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efdef2-8865-49eb-87e7-3a3036e7ca2c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:43.045316 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 195/362] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:43.046086 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4be8f-d7e3-4e26-addf-64566e5a228d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:43.147550 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efdef2-8865-49eb-87e7-3a3036e7ca2c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:43.174457 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4be8f-d7e3-4e26-addf-64566e5a228d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:43.211363 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:43.212573 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24bb6c-0eb3-43e5-a9ed-421531b5588a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:43.214086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24bb6c-0eb3-43e5-a9ed-421531b5588a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:43.214411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24bb6c-0eb3-43e5-a9ed-421531b5588a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:43.214685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24bb6c-0eb3-43e5-a9ed-421531b5588a None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:43.215905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a24bb6c-0eb3-43e5-a9ed-421531b5588a None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:43.217440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a24bb6c-0eb3-43e5-a9ed-421531b5588a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:43.219889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 168/363] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:43 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:43.220508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88687d9b-6bc6-4043-a0e1-ef007733eea8 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:43.220725 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88687d9b-6bc6-4043-a0e1-ef007733eea8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:43.221000 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88687d9b-6bc6-4043-a0e1-ef007733eea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:43.222869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38f0f0-c7f5-4347-ae19-6914630691c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:43.223886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38f0f0-c7f5-4347-ae19-6914630691c2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:43.224189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38f0f0-c7f5-4347-ae19-6914630691c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:43.224457 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38f0f0-c7f5-4347-ae19-6914630691c2 None admin] PATH_INFO: `/v3/groups/9758043b2c3a4673b089931b98c328e4` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:43.225645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38f0f0-c7f5-4347-ae19-6914630691c2 None admin] RBAC: Authorizing `identity:get_group(group_id=9758043b2c3a4673b089931b98c328e4)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:43.227967 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38f0f0-c7f5-4347-ae19-6914630691c2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:43.229838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 196/364] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jul 12 13:28:43 2022] GET /identity/v3/groups/9758043b2c3a4673b089931b98c328e4 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:43.230532 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38f0f0-c7f5-4347-ae19-6914630691c2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:43.242603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-88687d9b-6bc6-4043-a0e1-ef007733eea8 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:43.243837 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-88687d9b-6bc6-4043-a0e1-ef007733eea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:43.299311 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:43.334114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 169/365] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:43.335383 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88687d9b-6bc6-4043-a0e1-ef007733eea8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:43.364965 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38f0f0-c7f5-4347-ae19-6914630691c2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:43.420735 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609de2e-bbc7-40cd-b511-a2763a777b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:43.421569 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609de2e-bbc7-40cd-b511-a2763a777b03 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:43.422148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609de2e-bbc7-40cd-b511-a2763a777b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:43.422148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609de2e-bbc7-40cd-b511-a2763a777b03 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:43.437710 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609de2e-bbc7-40cd-b511-a2763a777b03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:43.446697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-8609de2e-bbc7-40cd-b511-a2763a777b03 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:43.446697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:43.446697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:43.446697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:43.446697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:43.446697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:43.446697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:43.446697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:43.446697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:43.446697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:43.446697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:43.446697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:43.446697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:43.446697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:43.446697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:43.446697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:43.447651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:43.447651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:43.447651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:43.447651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:43.447651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:43.447651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:43.447651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:43.447651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:43.447651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:43.447651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:43.447651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:43.447651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:43.447651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:43.447651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:43.447651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:43.447651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:43.448531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:43.448531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:43.448531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:43.448531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:43.448531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:43.448531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:43.448531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:43.448531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:43.448531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:43.448531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:43.448531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:43.448531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:43.448531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 197/366] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:28:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:43.448531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609de2e-bbc7-40cd-b511-a2763a777b03 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:43.471173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88687d9b-6bc6-4043-a0e1-ef007733eea8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:43.516523 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8d131-184b-4b9c-9013-68adc445386d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:43.518372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8d131-184b-4b9c-9013-68adc445386d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:43.518372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8d131-184b-4b9c-9013-68adc445386d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:43.518767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8d131-184b-4b9c-9013-68adc445386d None admin] PATH_INFO: `/v3/groups/9758043b2c3a4673b089931b98c328e4` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:43.522551 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e8d131-184b-4b9c-9013-68adc445386d None admin] RBAC: Authorizing `identity:get_group(group_id=9758043b2c3a4673b089931b98c328e4)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:43.524794 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e8d131-184b-4b9c-9013-68adc445386d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:43.527534 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 170/367] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jul 12 13:28:43 2022] GET /identity/v3/groups/9758043b2c3a4673b089931b98c328e4 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:43.530037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8d131-184b-4b9c-9013-68adc445386d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:43.530680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:43.589912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:43.590586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609de2e-bbc7-40cd-b511-a2763a777b03 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:43.640907 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8254c-6990-4386-8645-389a6b5895e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:43.641827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8254c-6990-4386-8645-389a6b5895e8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:43.642143 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8254c-6990-4386-8645-389a6b5895e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:43.642994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8254c-6990-4386-8645-389a6b5895e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:43.643651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8254c-6990-4386-8645-389a6b5895e8 None admin] RBAC: Adding query filter params (group.id=9758043b2c3a4673b089931b98c328e4, role.id=ed842da869724411a5b355cd12f1c5c8, scope.project.id=ae719f7141f34603811594a41497e87c) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:43.643967 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8254c-6990-4386-8645-389a6b5895e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:43.649315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8254c-6990-4386-8645-389a6b5895e8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:43.654438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 198/368] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jul 12 13:28:43 2022] GET /identity/v3/role_assignments?group.id=9758043b2c3a4673b089931b98c328e4&scope.project.id=ae719f7141f34603811594a41497e87c&role.id=ed842da869724411a5b355cd12f1c5c8 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:43.655418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8254c-6990-4386-8645-389a6b5895e8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:43.659274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8d131-184b-4b9c-9013-68adc445386d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:43.714430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a689937-37a3-4cbe-98de-0a9cddcb8f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:43.716173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a689937-37a3-4cbe-98de-0a9cddcb8f14 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:43.716706 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a689937-37a3-4cbe-98de-0a9cddcb8f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:43.717601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a689937-37a3-4cbe-98de-0a9cddcb8f14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:43.736278 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a689937-37a3-4cbe-98de-0a9cddcb8f14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:43.749477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-2a689937-37a3-4cbe-98de-0a9cddcb8f14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:43.749477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:43.749477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:43.749477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:43.749477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:43.749477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:43.749477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:43.749477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:43.749477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:43.749477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:43.749477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:43.749477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:43.749477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:43.749477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:43.749477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:43.749477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:43.750512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:43.750512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:43.750512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:43.750512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:43.750512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:43.750512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:43.750512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:43.750512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:43.750512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:43.750512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:43.750512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:43.750512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:43.750512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:43.750512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:43.750512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:43.750512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:43.751323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:43.751323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:43.751323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:43.751323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:43.751323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:43.751323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:43.751323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:43.751323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:43.751323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:43.751323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:43.751323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:43.751323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:43.755016 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 171/369] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:28:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:43.756318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a689937-37a3-4cbe-98de-0a9cddcb8f14 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:43.815121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8254c-6990-4386-8645-389a6b5895e8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:43.845627 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:43.863725 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbc2d1-fe25-4a8b-b39a-d3b51ab9e78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:43.863725 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbc2d1-fe25-4a8b-b39a-d3b51ab9e78c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:43.863725 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbc2d1-fe25-4a8b-b39a-d3b51ab9e78c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:43.863725 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbc2d1-fe25-4a8b-b39a-d3b51ab9e78c None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:43.864616 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbc2d1-fe25-4a8b-b39a-d3b51ab9e78c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:43.864729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbc2d1-fe25-4a8b-b39a-d3b51ab9e78c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:43.866642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbc2d1-fe25-4a8b-b39a-d3b51ab9e78c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:43.877913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 199/370] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:28:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:43.881268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbc2d1-fe25-4a8b-b39a-d3b51ab9e78c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:43.908710 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a689937-37a3-4cbe-98de-0a9cddcb8f14 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:43.914418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:43.967525 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62ea18-e5b1-46d0-aa48-ad0eea4b0dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:43.968211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62ea18-e5b1-46d0-aa48-ad0eea4b0dec None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:43.968501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62ea18-e5b1-46d0-aa48-ad0eea4b0dec None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:43.968790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62ea18-e5b1-46d0-aa48-ad0eea4b0dec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:43.970874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62ea18-e5b1-46d0-aa48-ad0eea4b0dec None admin] RBAC: Adding query filter params (group.id=9758043b2c3a4673b089931b98c328e4, role.id=ed842da869724411a5b355cd12f1c5c8, scope.project.id=abaed0828ac34adb801becefa86d65b3) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:43.971010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62ea18-e5b1-46d0-aa48-ad0eea4b0dec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:43.972469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62ea18-e5b1-46d0-aa48-ad0eea4b0dec None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:43.983832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 172/371] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jul 12 13:28:43 2022] GET /identity/v3/role_assignments?group.id=9758043b2c3a4673b089931b98c328e4&scope.project.id=abaed0828ac34adb801becefa86d65b3&role.id=ed842da869724411a5b355cd12f1c5c8 => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:43.983832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62ea18-e5b1-46d0-aa48-ad0eea4b0dec None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:44.015712 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbc2d1-fe25-4a8b-b39a-d3b51ab9e78c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:44.068059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319a33c-e339-41a6-83b6-57ab89d39648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:44.069171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319a33c-e339-41a6-83b6-57ab89d39648 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:44.069666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319a33c-e339-41a6-83b6-57ab89d39648 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:44.070170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319a33c-e339-41a6-83b6-57ab89d39648 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:44.071804 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319a33c-e339-41a6-83b6-57ab89d39648 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:44.072357 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319a33c-e339-41a6-83b6-57ab89d39648 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:44.074022 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319a33c-e339-41a6-83b6-57ab89d39648 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:44.090975 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 200/372] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:44.093190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319a33c-e339-41a6-83b6-57ab89d39648 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:44.151645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:44.219394 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319a33c-e339-41a6-83b6-57ab89d39648 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:44.266112 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaae785-279b-4b0f-984f-0264e09ff51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:44.267312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaae785-279b-4b0f-984f-0264e09ff51b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:44.267815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaae785-279b-4b0f-984f-0264e09ff51b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:44.268306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaae785-279b-4b0f-984f-0264e09ff51b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:44.268429 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62ea18-e5b1-46d0-aa48-ad0eea4b0dec None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:44.327712 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53446580-61ad-44fd-b83f-81b65aa58899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:44.328495 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53446580-61ad-44fd-b83f-81b65aa58899 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:44.328791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53446580-61ad-44fd-b83f-81b65aa58899 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:44.329070 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53446580-61ad-44fd-b83f-81b65aa58899 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:44.372143 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaae785-279b-4b0f-984f-0264e09ff51b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:44.373713 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaae785-279b-4b0f-984f-0264e09ff51b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:44.394677 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:44.453664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53446580-61ad-44fd-b83f-81b65aa58899 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:44.455102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53446580-61ad-44fd-b83f-81b65aa58899 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:44.464740 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:44.549806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 201/373] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:44.555276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaae785-279b-4b0f-984f-0264e09ff51b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:44.617733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 173/374] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:44.623738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53446580-61ad-44fd-b83f-81b65aa58899 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:44.665656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaae785-279b-4b0f-984f-0264e09ff51b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:44.703394 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37f5b4-de7e-4441-954e-e1b76d804f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:44.704347 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37f5b4-de7e-4441-954e-e1b76d804f36 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:44.704574 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37f5b4-de7e-4441-954e-e1b76d804f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:44.704852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37f5b4-de7e-4441-954e-e1b76d804f36 None admin] PATH_INFO: `/v3/projects/8e45bcca33094251b8a3ed60293c2eb5` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:44.706135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37f5b4-de7e-4441-954e-e1b76d804f36 None admin] RBAC: Authorizing `identity:get_project(project_id=8e45bcca33094251b8a3ed60293c2eb5)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:44.707623 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37f5b4-de7e-4441-954e-e1b76d804f36 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:44.709650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 202/375] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:44 2022] GET /identity/v3/projects/8e45bcca33094251b8a3ed60293c2eb5 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:44.714944 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37f5b4-de7e-4441-954e-e1b76d804f36 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:44.733890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:44.749732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:44.808284 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53446580-61ad-44fd-b83f-81b65aa58899 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:44.865991 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874566d-536a-4c4b-9f48-c0dce2fb7063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:44.866882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874566d-536a-4c4b-9f48-c0dce2fb7063 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:44.867164 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874566d-536a-4c4b-9f48-c0dce2fb7063 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:44.867437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874566d-536a-4c4b-9f48-c0dce2fb7063 None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:44.868727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874566d-536a-4c4b-9f48-c0dce2fb7063 None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:44.870309 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874566d-536a-4c4b-9f48-c0dce2fb7063 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:44.872444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 174/376] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:44 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:44.874340 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a0869-1f9d-4771-862e-369ff0d57334 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:44.874622 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a0869-1f9d-4771-862e-369ff0d57334 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:44.874893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a0869-1f9d-4771-862e-369ff0d57334 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:44.876377 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 175/377] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:44.882422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a0869-1f9d-4771-862e-369ff0d57334 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:44.892674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37f5b4-de7e-4441-954e-e1b76d804f36 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:44.939483 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3788f-d68a-4b5e-a439-d5bc97050044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:44.940351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3788f-d68a-4b5e-a439-d5bc97050044 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:44.940688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3788f-d68a-4b5e-a439-d5bc97050044 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:44.940961 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3788f-d68a-4b5e-a439-d5bc97050044 None admin] PATH_INFO: `/v3/groups/799b2f0fc52e4bfa99efa19842fb0852` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:44.948717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3788f-d68a-4b5e-a439-d5bc97050044 None admin] RBAC: Authorizing `identity:get_group(group_id=799b2f0fc52e4bfa99efa19842fb0852)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:44.950330 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3788f-d68a-4b5e-a439-d5bc97050044 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:44.953046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 203/378] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jul 12 13:28:44 2022] GET /identity/v3/groups/799b2f0fc52e4bfa99efa19842fb0852 => generated 229 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:44.958360 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033dde30-9de0-4e77-bb7c-910a3c51470a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:44.958610 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033dde30-9de0-4e77-bb7c-910a3c51470a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:44.958850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033dde30-9de0-4e77-bb7c-910a3c51470a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:44.978272 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-033dde30-9de0-4e77-bb7c-910a3c51470a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:44.979925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-033dde30-9de0-4e77-bb7c-910a3c51470a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:45.001114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a0869-1f9d-4771-862e-369ff0d57334 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:45.057329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cfd60-f9ca-4696-9e81-03a561ba3fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:45.059131 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cfd60-f9ca-4696-9e81-03a561ba3fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:45.059422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cfd60-f9ca-4696-9e81-03a561ba3fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:45.059691 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cfd60-f9ca-4696-9e81-03a561ba3fa8 None admin] PATH_INFO: `/v3/groups/9758043b2c3a4673b089931b98c328e4` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:45.061768 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cfd60-f9ca-4696-9e81-03a561ba3fa8 None admin] RBAC: Authorizing `identity:get_group(group_id=9758043b2c3a4673b089931b98c328e4)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:45.063297 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cfd60-f9ca-4696-9e81-03a561ba3fa8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:45.067909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 176/379] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jul 12 13:28:44 2022] GET /identity/v3/groups/9758043b2c3a4673b089931b98c328e4 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:45.069589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cfd60-f9ca-4696-9e81-03a561ba3fa8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:45.078670 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:45.099576 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 204/380] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:45.103028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033dde30-9de0-4e77-bb7c-910a3c51470a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:45.122320 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:45.212261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cfd60-f9ca-4696-9e81-03a561ba3fa8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:45.219382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033dde30-9de0-4e77-bb7c-910a3c51470a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:45.249374 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf21e5-d8d2-4047-8541-0a5dd1d2497b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:45.250120 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf21e5-d8d2-4047-8541-0a5dd1d2497b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:45.250399 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf21e5-d8d2-4047-8541-0a5dd1d2497b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:45.250679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf21e5-d8d2-4047-8541-0a5dd1d2497b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:45.251488 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf21e5-d8d2-4047-8541-0a5dd1d2497b None admin] RBAC: Adding query filter params (group.id=799b2f0fc52e4bfa99efa19842fb0852, role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=8e45bcca33094251b8a3ed60293c2eb5) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:45.251964 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf21e5-d8d2-4047-8541-0a5dd1d2497b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:45.253480 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf21e5-d8d2-4047-8541-0a5dd1d2497b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:45.260627 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 177/381] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jul 12 13:28:45 2022] GET /identity/v3/role_assignments?group.id=799b2f0fc52e4bfa99efa19842fb0852&scope.project.id=8e45bcca33094251b8a3ed60293c2eb5&role.id=864a6966630544e8bd68bcafba905c6f => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:45.261128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d769f5-a75a-404d-a4de-4b0dac771cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:45.261485 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf21e5-d8d2-4047-8541-0a5dd1d2497b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:45.262337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d769f5-a75a-404d-a4de-4b0dac771cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:45.262805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d769f5-a75a-404d-a4de-4b0dac771cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:45.263284 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d769f5-a75a-404d-a4de-4b0dac771cc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:45.264364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d769f5-a75a-404d-a4de-4b0dac771cc5 None admin] RBAC: Adding query filter params (group.id=9758043b2c3a4673b089931b98c328e4, role.id=dbe30950eded4c9ab2bcc0f7545b9bae, scope.project.id=ae719f7141f34603811594a41497e87c) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:45.264887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d769f5-a75a-404d-a4de-4b0dac771cc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:45.266583 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d769f5-a75a-404d-a4de-4b0dac771cc5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:45.278391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 205/382] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jul 12 13:28:45 2022] GET /identity/v3/role_assignments?group.id=9758043b2c3a4673b089931b98c328e4&scope.project.id=ae719f7141f34603811594a41497e87c&role.id=dbe30950eded4c9ab2bcc0f7545b9bae => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:45.379454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:45.429404 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf21e5-d8d2-4047-8541-0a5dd1d2497b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:45.482352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e034526-5144-4e82-be73-d8885e8500df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:45.484401 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e034526-5144-4e82-be73-d8885e8500df None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:45.484913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e034526-5144-4e82-be73-d8885e8500df None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:45.485420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e034526-5144-4e82-be73-d8885e8500df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:45.499851 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e034526-5144-4e82-be73-d8885e8500df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:45.508742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-0e034526-5144-4e82-be73-d8885e8500df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:45.508742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:45.508742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:45.508742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:45.508742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:45.508742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:45.508742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:45.508742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:45.508742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:45.508742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:45.508742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:45.508742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:45.508742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:45.508742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:45.508742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:45.508742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:45.509696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:45.509696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:45.509696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:45.509696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:45.509696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:45.509696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:45.509696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:45.509696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:45.509696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:45.509696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:45.509696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:45.509696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:45.509696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:45.509696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:45.509696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:45.509696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:45.510537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:45.510537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:45.510537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:45.510537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:45.510537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:45.510537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:45.510537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:45.510537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:45.510537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:45.510537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:45.510537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:45.510537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:45.510537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 178/383] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:28:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:45.516140 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d769f5-a75a-404d-a4de-4b0dac771cc5 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:45.595468 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:45.636282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d769f5-a75a-404d-a4de-4b0dac771cc5 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:45.678474 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cbb92-b79f-4d6f-8adf-489473751cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:45.679516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cbb92-b79f-4d6f-8adf-489473751cfb None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:45.680095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cbb92-b79f-4d6f-8adf-489473751cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:45.680580 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cbb92-b79f-4d6f-8adf-489473751cfb None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:45.681544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cbb92-b79f-4d6f-8adf-489473751cfb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:45.682131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cbb92-b79f-4d6f-8adf-489473751cfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:45.683731 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cbb92-b79f-4d6f-8adf-489473751cfb None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:45.697406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 206/384] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:28:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:45.702684 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e034526-5144-4e82-be73-d8885e8500df None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:45.831758 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e034526-5144-4e82-be73-d8885e8500df None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:45.867752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:45.875863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794581a-50f9-473a-b61a-6ad6264e2ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:45.877114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794581a-50f9-473a-b61a-6ad6264e2ccc None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:45.878309 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794581a-50f9-473a-b61a-6ad6264e2ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:45.878687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794581a-50f9-473a-b61a-6ad6264e2ccc None admin] PATH_INFO: `/v3/groups/9758043b2c3a4673b089931b98c328e4` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:45.880029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794581a-50f9-473a-b61a-6ad6264e2ccc None admin] RBAC: Authorizing `identity:get_group(group_id=9758043b2c3a4673b089931b98c328e4)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:45.881846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794581a-50f9-473a-b61a-6ad6264e2ccc None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:45.883967 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 179/385] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jul 12 13:28:45 2022] GET /identity/v3/groups/9758043b2c3a4673b089931b98c328e4 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:45.890424 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cbb92-b79f-4d6f-8adf-489473751cfb None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:46.030277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cbb92-b79f-4d6f-8adf-489473751cfb None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:46.072412 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a53cc-b4ae-41cf-9fe6-555ea2ccf3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:46.073288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a53cc-b4ae-41cf-9fe6-555ea2ccf3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:46.073568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a53cc-b4ae-41cf-9fe6-555ea2ccf3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:46.073843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a53cc-b4ae-41cf-9fe6-555ea2ccf3b7 None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:46.075089 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a53cc-b4ae-41cf-9fe6-555ea2ccf3b7 None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:46.077075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a53cc-b4ae-41cf-9fe6-555ea2ccf3b7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:46.079113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 207/386] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:45 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:46.084453 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794581a-50f9-473a-b61a-6ad6264e2ccc None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:46.207914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794581a-50f9-473a-b61a-6ad6264e2ccc None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:46.249898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3273549-965b-4c10-a410-54435d006fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:46.250819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3273549-965b-4c10-a410-54435d006fb1 None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:46.251096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3273549-965b-4c10-a410-54435d006fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:46.251361 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3273549-965b-4c10-a410-54435d006fb1 None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3/groups/9758043b2c3a4673b089931b98c328e4/roles/dbe30950eded4c9ab2bcc0f7545b9bae` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:46.256561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45de991-45dc-45a9-99f4-f08a4fd02756 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:46.256740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3273549-965b-4c10-a410-54435d006fb1 None admin] RBAC: Authorizing `identity:create_grant(project_id=abaed0828ac34adb801becefa86d65b3, group_id=9758043b2c3a4673b089931b98c328e4, role_id=dbe30950eded4c9ab2bcc0f7545b9bae)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:46.257094 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45de991-45dc-45a9-99f4-f08a4fd02756 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:46.257467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45de991-45dc-45a9-99f4-f08a4fd02756 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:46.258177 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3273549-965b-4c10-a410-54435d006fb1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:46.259094 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 208/387] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:46.268282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4f63e-6390-4fc8-b79b-c169d835be79 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:46.268593 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4f63e-6390-4fc8-b79b-c169d835be79 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:46.269054 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4f63e-6390-4fc8-b79b-c169d835be79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:46.281811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:46.286478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 180/388] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Jul 12 13:28:46 2022] PUT /identity/v3/projects/abaed0828ac34adb801becefa86d65b3/groups/9758043b2c3a4673b089931b98c328e4/roles/dbe30950eded4c9ab2bcc0f7545b9bae => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:46.301104 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-29b4f63e-6390-4fc8-b79b-c169d835be79 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:46.302684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-29b4f63e-6390-4fc8-b79b-c169d835be79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:46.384497 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 209/389] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:46.409613 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3273549-965b-4c10-a410-54435d006fb1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:46.487363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:46.543362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3273549-965b-4c10-a410-54435d006fb1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:46.591044 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf2357-774e-4f2b-8b5e-b4132469296f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:46.592959 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf2357-774e-4f2b-8b5e-b4132469296f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:46.593424 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf2357-774e-4f2b-8b5e-b4132469296f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:46.593891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf2357-774e-4f2b-8b5e-b4132469296f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:46.609618 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf2357-774e-4f2b-8b5e-b4132469296f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:46.617813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-b7bf2357-774e-4f2b-8b5e-b4132469296f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:46.617813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:46.617813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:46.617813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:46.617813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:46.617813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:46.617813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:46.617813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:46.617813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:46.617813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:46.617813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:46.617813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:46.617813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:46.617813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:46.617813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:46.617813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:46.618720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:46.618720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:46.618720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:46.618720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:46.618720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:46.618720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:46.618720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:46.618720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:46.618720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:46.618720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:46.618720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:46.618720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:46.618720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:46.618720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:46.618720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:46.618720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:46.619496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:46.619496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:46.619496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:46.619496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:46.619496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:46.619496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:46.619496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:46.619496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:46.619496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:46.619496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:46.619496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:46.619496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:46.621916 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 181/390] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:46.628297 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4f63e-6390-4fc8-b79b-c169d835be79 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:46.720832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d2690-4d5b-4441-9ac4-b5c17fbf169e None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:46.721691 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d2690-4d5b-4441-9ac4-b5c17fbf169e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:46.722150 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d2690-4d5b-4441-9ac4-b5c17fbf169e None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:46.723649 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 182/391] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:28:46.731369 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44230226-e7c8-4606-8a49-08e7fa575a0b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:46.731941 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44230226-e7c8-4606-8a49-08e7fa575a0b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:46.732461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44230226-e7c8-4606-8a49-08e7fa575a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:46.747818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4f63e-6390-4fc8-b79b-c169d835be79 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:46.754165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-44230226-e7c8-4606-8a49-08e7fa575a0b None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:46.755633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-44230226-e7c8-4606-8a49-08e7fa575a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:46.803368 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:46.809545 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c31e3-e244-4c6c-b255-88db6cbb117c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:46.810254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c31e3-e244-4c6c-b255-88db6cbb117c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:46.810564 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c31e3-e244-4c6c-b255-88db6cbb117c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:46.810838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c31e3-e244-4c6c-b255-88db6cbb117c None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:46.811644 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c31e3-e244-4c6c-b255-88db6cbb117c None admin] RBAC: Adding query filter params (name=member) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:46.811951 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c31e3-e244-4c6c-b255-88db6cbb117c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:46.813728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c31e3-e244-4c6c-b255-88db6cbb117c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:46.824147 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 210/392] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:46.832220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c31e3-e244-4c6c-b255-88db6cbb117c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:46.860394 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 183/393] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:46.943031 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44230226-e7c8-4606-8a49-08e7fa575a0b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:46.973030 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c31e3-e244-4c6c-b255-88db6cbb117c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:47.012098 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7cb0b-fb3c-4be2-bc61-26374eaf2802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:47.013048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7cb0b-fb3c-4be2-bc61-26374eaf2802 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:47.013334 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7cb0b-fb3c-4be2-bc61-26374eaf2802 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:47.014188 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7cb0b-fb3c-4be2-bc61-26374eaf2802 None admin] PATH_INFO: `/v3/groups/9758043b2c3a4673b089931b98c328e4` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:47.015684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7cb0b-fb3c-4be2-bc61-26374eaf2802 None admin] RBAC: Authorizing `identity:get_group(group_id=9758043b2c3a4673b089931b98c328e4)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:47.017319 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7cb0b-fb3c-4be2-bc61-26374eaf2802 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:47.019451 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 211/394] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jul 12 13:28:46 2022] GET /identity/v3/groups/9758043b2c3a4673b089931b98c328e4 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:47.025237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7cb0b-fb3c-4be2-bc61-26374eaf2802 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:47.064684 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:47.098447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:47.135028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7cb0b-fb3c-4be2-bc61-26374eaf2802 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:47.173446 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b2731-b08b-4c1e-a3f7-e5ad1ee78afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:47.175152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b2731-b08b-4c1e-a3f7-e5ad1ee78afd None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:47.175776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b2731-b08b-4c1e-a3f7-e5ad1ee78afd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:47.176392 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b2731-b08b-4c1e-a3f7-e5ad1ee78afd None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:47.178384 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b2731-b08b-4c1e-a3f7-e5ad1ee78afd None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:47.180097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b2731-b08b-4c1e-a3f7-e5ad1ee78afd None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:47.182316 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 212/395] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:47 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:47.188605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b2731-b08b-4c1e-a3f7-e5ad1ee78afd None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:47.224266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44230226-e7c8-4606-8a49-08e7fa575a0b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:47.288500 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221bfe6-711a-4437-a14f-e5782f574460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:47.289307 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221bfe6-711a-4437-a14f-e5782f574460 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:47.289586 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221bfe6-711a-4437-a14f-e5782f574460 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:47.289866 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221bfe6-711a-4437-a14f-e5782f574460 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:47.297048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b2731-b08b-4c1e-a3f7-e5ad1ee78afd None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:47.305489 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221bfe6-711a-4437-a14f-e5782f574460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:47.321653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-e221bfe6-711a-4437-a14f-e5782f574460 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:47.321653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:47.321653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:47.321653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:47.321653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:47.321653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:47.321653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:47.321653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:47.321653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:47.321653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:47.321653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:47.321653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:47.321653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:47.321653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:47.321653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:47.321653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:47.322640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:47.322640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:47.322640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:47.322640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:47.322640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:47.322640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:47.322640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:47.322640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:47.322640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:47.322640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:47.322640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:47.322640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:47.322640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:47.322640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:47.322640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:47.322640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:47.323420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:47.323420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:47.323420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:47.323420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:47.323420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:47.323420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:47.323420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:47.323420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:47.323420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:47.323420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:47.323420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:47.323420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:47.323420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 184/396] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:47.334936 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221bfe6-711a-4437-a14f-e5782f574460 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:47.342364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273715c-3de2-4fb6-b29f-374509260c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:47.343637 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273715c-3de2-4fb6-b29f-374509260c5a None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:47.344131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273715c-3de2-4fb6-b29f-374509260c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:47.344702 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273715c-3de2-4fb6-b29f-374509260c5a None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c/groups/9758043b2c3a4673b089931b98c328e4/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:47.346997 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273715c-3de2-4fb6-b29f-374509260c5a None admin] RBAC: Authorizing `identity:create_grant(project_id=ae719f7141f34603811594a41497e87c, group_id=9758043b2c3a4673b089931b98c328e4, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:47.348863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273715c-3de2-4fb6-b29f-374509260c5a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:47.354270 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:47.371093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 213/397] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Jul 12 13:28:47 2022] PUT /identity/v3/projects/ae719f7141f34603811594a41497e87c/groups/9758043b2c3a4673b089931b98c328e4/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:47.472020 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221bfe6-711a-4437-a14f-e5782f574460 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:47.480081 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:47.515524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cda02-b178-4c3d-b5fc-9ac4fc768069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:47.516621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cda02-b178-4c3d-b5fc-9ac4fc768069 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:47.517122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cda02-b178-4c3d-b5fc-9ac4fc768069 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:47.517643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cda02-b178-4c3d-b5fc-9ac4fc768069 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:47.518661 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106cda02-b178-4c3d-b5fc-9ac4fc768069 None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:47.519256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106cda02-b178-4c3d-b5fc-9ac4fc768069 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:47.520895 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106cda02-b178-4c3d-b5fc-9ac4fc768069 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:47.532689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 185/398] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:47.538754 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273715c-3de2-4fb6-b29f-374509260c5a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:47.660745 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273715c-3de2-4fb6-b29f-374509260c5a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:47.698110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714baaa-9616-47b0-829a-4967ffdde4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:47.699434 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714baaa-9616-47b0-829a-4967ffdde4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:47.699930 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714baaa-9616-47b0-829a-4967ffdde4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:47.700400 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714baaa-9616-47b0-829a-4967ffdde4a3 None admin] PATH_INFO: `/v3/groups/9758043b2c3a4673b089931b98c328e4` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:47.701942 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714baaa-9616-47b0-829a-4967ffdde4a3 None admin] RBAC: Authorizing `identity:get_group(group_id=9758043b2c3a4673b089931b98c328e4)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:47.703672 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714baaa-9616-47b0-829a-4967ffdde4a3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:47.705817 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 214/399] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jul 12 13:28:47 2022] GET /identity/v3/groups/9758043b2c3a4673b089931b98c328e4 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:47.711513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cda02-b178-4c3d-b5fc-9ac4fc768069 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:47.831442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cda02-b178-4c3d-b5fc-9ac4fc768069 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:47.882031 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbe012-f331-4983-961c-ec485cef46cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:47.883356 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbe012-f331-4983-961c-ec485cef46cb None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:47.883879 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbe012-f331-4983-961c-ec485cef46cb None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:47.884346 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbe012-f331-4983-961c-ec485cef46cb None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:47.885733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cbe012-f331-4983-961c-ec485cef46cb None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:47.887401 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cbe012-f331-4983-961c-ec485cef46cb None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:47.889768 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 186/400] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:47 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:47.895685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714baaa-9616-47b0-829a-4967ffdde4a3 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:47.961058 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:48.003214 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714baaa-9616-47b0-829a-4967ffdde4a3 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:48.039687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c28dc-082b-4e58-b473-d4bff2812507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:48.040564 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c28dc-082b-4e58-b473-d4bff2812507 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:48.040842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c28dc-082b-4e58-b473-d4bff2812507 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:48.041113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c28dc-082b-4e58-b473-d4bff2812507 None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3/groups/9758043b2c3a4673b089931b98c328e4/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:48.043197 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c28dc-082b-4e58-b473-d4bff2812507 None admin] RBAC: Authorizing `identity:create_grant(project_id=abaed0828ac34adb801becefa86d65b3, group_id=9758043b2c3a4673b089931b98c328e4, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:48.044655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c28dc-082b-4e58-b473-d4bff2812507 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:48.061808 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 215/401] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Jul 12 13:28:47 2022] PUT /identity/v3/projects/abaed0828ac34adb801becefa86d65b3/groups/9758043b2c3a4673b089931b98c328e4/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:48.191962 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e24b2e-12d9-479b-841e-85143c66fc24 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:48.192361 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e24b2e-12d9-479b-841e-85143c66fc24 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:48.192546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e24b2e-12d9-479b-841e-85143c66fc24 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:48.194041 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 187/402] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:28:48.199889 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7287c-69fd-4e85-b57a-75de4deeb382 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:48.200187 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7287c-69fd-4e85-b57a-75de4deeb382 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:48.200472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7287c-69fd-4e85-b57a-75de4deeb382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:48.219159 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-27d7287c-69fd-4e85-b57a-75de4deeb382 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:48.220396 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-27d7287c-69fd-4e85-b57a-75de4deeb382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:48.302091 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 216/403] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:48.322112 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e24b2e-12d9-479b-841e-85143c66fc24 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:48.361923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:48.439187 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e24b2e-12d9-479b-841e-85143c66fc24 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:48.483110 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc81501-de30-40ce-93d8-309e1d717305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:48.483852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc81501-de30-40ce-93d8-309e1d717305 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:48.484132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc81501-de30-40ce-93d8-309e1d717305 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:48.484406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc81501-de30-40ce-93d8-309e1d717305 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:48.501715 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc81501-de30-40ce-93d8-309e1d717305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:48.510733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-abc81501-de30-40ce-93d8-309e1d717305 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:48.510733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:48.510733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:48.510733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:48.510733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:48.510733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:48.510733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:48.510733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:48.510733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:48.510733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:48.510733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:48.510733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:48.510733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:48.510733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:48.510733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:48.510733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:48.511688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:48.511688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:48.511688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:48.511688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:48.511688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:48.511688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:48.511688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:48.511688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:48.511688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:48.511688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:48.511688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:48.511688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:48.511688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:48.511688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:48.511688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:48.511688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:48.512471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:48.512471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:48.512471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:48.512471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:48.512471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:48.512471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:48.512471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:48.512471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:48.512471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:48.512471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:48.512471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:48.512471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:48.517097 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 188/404] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:28:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:48.518525 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7287c-69fd-4e85-b57a-75de4deeb382 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:48.530739 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c3f7c-dc0c-4a04-b78e-c35bee425b1e None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:48.531045 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c3f7c-dc0c-4a04-b78e-c35bee425b1e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:48.531304 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c3f7c-dc0c-4a04-b78e-c35bee425b1e None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:48.533415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 189/405] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:48.541903 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed4b91-16f8-42f5-9463-8f2703bd535a None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:48.542202 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed4b91-16f8-42f5-9463-8f2703bd535a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:48.542545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed4b91-16f8-42f5-9463-8f2703bd535a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:48.568756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-2eed4b91-16f8-42f5-9463-8f2703bd535a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:48.569986 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2eed4b91-16f8-42f5-9463-8f2703bd535a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:48.604321 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:48.695739 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7287c-69fd-4e85-b57a-75de4deeb382 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:48.700428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 190/406] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:48.732477 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27479937-2efb-418f-8082-7b12e8d02661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:48.733249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27479937-2efb-418f-8082-7b12e8d02661 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:48.733480 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27479937-2efb-418f-8082-7b12e8d02661 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:48.733794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27479937-2efb-418f-8082-7b12e8d02661 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:48.734628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27479937-2efb-418f-8082-7b12e8d02661 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:48.734937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27479937-2efb-418f-8082-7b12e8d02661 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:48.736395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27479937-2efb-418f-8082-7b12e8d02661 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:48.745840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed4b91-16f8-42f5-9463-8f2703bd535a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:48.746961 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 217/407] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:28:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:48.761257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27479937-2efb-418f-8082-7b12e8d02661 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:48.794340 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:48.923791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27479937-2efb-418f-8082-7b12e8d02661 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:48.944316 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:48.947238 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed4b91-16f8-42f5-9463-8f2703bd535a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:48.968980 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961f385-a057-4e76-882d-84891e0f8de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:48.969854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961f385-a057-4e76-882d-84891e0f8de7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:48.970181 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961f385-a057-4e76-882d-84891e0f8de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:48.970450 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961f385-a057-4e76-882d-84891e0f8de7 None admin] PATH_INFO: `/v3/groups/9758043b2c3a4673b089931b98c328e4` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:48.971753 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961f385-a057-4e76-882d-84891e0f8de7 None admin] RBAC: Authorizing `identity:get_group(group_id=9758043b2c3a4673b089931b98c328e4)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:48.973278 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961f385-a057-4e76-882d-84891e0f8de7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:48.975300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 218/408] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jul 12 13:28:48 2022] GET /identity/v3/groups/9758043b2c3a4673b089931b98c328e4 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:48.976616 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a614b71-6d88-4747-b1e6-2829d8ccd873 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:48.976899 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a614b71-6d88-4747-b1e6-2829d8ccd873 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:48.977155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a614b71-6d88-4747-b1e6-2829d8ccd873 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:48.983080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 219/409] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:48.984043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a614b71-6d88-4747-b1e6-2829d8ccd873 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:48.994122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d401298-25bd-41a3-a798-698b8145a8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:48.994962 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d401298-25bd-41a3-a798-698b8145a8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:48.995627 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d401298-25bd-41a3-a798-698b8145a8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:48.995912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d401298-25bd-41a3-a798-698b8145a8c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:49.009916 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d401298-25bd-41a3-a798-698b8145a8c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:49.020393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-8d401298-25bd-41a3-a798-698b8145a8c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:49.020393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:49.020393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:49.020393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:49.020393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:49.020393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:49.020393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:49.020393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:49.020393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:49.020393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:49.020393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:49.020393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:49.020393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:49.020393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:49.020393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:49.020393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:49.021517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:49.021517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:49.021517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:49.021517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:49.021517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:49.021517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:49.021517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:49.021517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:49.021517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:49.021517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:49.021517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:49.021517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:49.021517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:49.021517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:49.021517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:49.021517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:49.022352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:49.022352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:49.022352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:49.022352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:49.022352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:49.022352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:49.022352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:49.022352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:49.022352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:49.022352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:49.022352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:49.022352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:49.022352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 191/410] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:28:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:49.023772 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12a11c-3800-4409-8f5e-6d2edc4cc868 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:49.023772 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12a11c-3800-4409-8f5e-6d2edc4cc868 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:49.024908 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12a11c-3800-4409-8f5e-6d2edc4cc868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:49.045337 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4e12a11c-3800-4409-8f5e-6d2edc4cc868 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:49.047039 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4e12a11c-3800-4409-8f5e-6d2edc4cc868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:49.103516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a614b71-6d88-4747-b1e6-2829d8ccd873 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:49.140427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973861c-3a7d-47ce-a108-d39c2b421e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:49.141293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973861c-3a7d-47ce-a108-d39c2b421e83 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:49.141578 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973861c-3a7d-47ce-a108-d39c2b421e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:49.141853 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973861c-3a7d-47ce-a108-d39c2b421e83 None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:49.142429 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 192/411] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:49.143123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973861c-3a7d-47ce-a108-d39c2b421e83 None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:49.143307 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12a11c-3800-4409-8f5e-6d2edc4cc868 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:49.144639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973861c-3a7d-47ce-a108-d39c2b421e83 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:49.150997 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 220/412] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:48 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:49.152187 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973861c-3a7d-47ce-a108-d39c2b421e83 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:49.175828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:49.216167 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:49.269999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12a11c-3800-4409-8f5e-6d2edc4cc868 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:49.277886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973861c-3a7d-47ce-a108-d39c2b421e83 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:49.305196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7e431-c4b9-4acc-af98-05f010c0c16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:49.306032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7e431-c4b9-4acc-af98-05f010c0c16b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:49.306413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7e431-c4b9-4acc-af98-05f010c0c16b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:49.306784 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7e431-c4b9-4acc-af98-05f010c0c16b None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:49.307707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7e431-c4b9-4acc-af98-05f010c0c16b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:49.308126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7e431-c4b9-4acc-af98-05f010c0c16b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:49.309587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7e431-c4b9-4acc-af98-05f010c0c16b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:49.317670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b64ef64-2ca4-48e3-b0bd-21f3b444bbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:49.318173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 193/413] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:49.319090 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7e431-c4b9-4acc-af98-05f010c0c16b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:49.319207 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64ef64-2ca4-48e3-b0bd-21f3b444bbd3 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:49.319622 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64ef64-2ca4-48e3-b0bd-21f3b444bbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:49.320039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64ef64-2ca4-48e3-b0bd-21f3b444bbd3 None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c/groups/9758043b2c3a4673b089931b98c328e4/roles/dbe30950eded4c9ab2bcc0f7545b9bae` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:49.323774 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b64ef64-2ca4-48e3-b0bd-21f3b444bbd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae719f7141f34603811594a41497e87c, group_id=9758043b2c3a4673b089931b98c328e4, role_id=dbe30950eded4c9ab2bcc0f7545b9bae)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:49.325448 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b64ef64-2ca4-48e3-b0bd-21f3b444bbd3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:49.343346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 221/414] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Jul 12 13:28:49 2022] PUT /identity/v3/projects/ae719f7141f34603811594a41497e87c/groups/9758043b2c3a4673b089931b98c328e4/roles/dbe30950eded4c9ab2bcc0f7545b9bae => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:49.347093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b64ef64-2ca4-48e3-b0bd-21f3b444bbd3 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:49.470335 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:49.529363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7e431-c4b9-4acc-af98-05f010c0c16b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:49.543799 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b64ef64-2ca4-48e3-b0bd-21f3b444bbd3 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:49.555042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:49.591073 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e62cc9-8104-4b91-819d-bb9d5933ebd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:49.592096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e62cc9-8104-4b91-819d-bb9d5933ebd5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:49.592399 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e62cc9-8104-4b91-819d-bb9d5933ebd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:49.592680 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e62cc9-8104-4b91-819d-bb9d5933ebd5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:49.604455 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9244a7-fa03-4d98-b844-a299c45d6fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:49.605448 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9244a7-fa03-4d98-b844-a299c45d6fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:49.605735 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9244a7-fa03-4d98-b844-a299c45d6fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:49.606081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9244a7-fa03-4d98-b844-a299c45d6fc2 None admin] PATH_INFO: `/v3/groups/799b2f0fc52e4bfa99efa19842fb0852` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:49.607923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9244a7-fa03-4d98-b844-a299c45d6fc2 None admin] RBAC: Authorizing `identity:get_group(group_id=799b2f0fc52e4bfa99efa19842fb0852)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:49.608555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e62cc9-8104-4b91-819d-bb9d5933ebd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:49.609391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9244a7-fa03-4d98-b844-a299c45d6fc2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:49.611342 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 222/415] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jul 12 13:28:49 2022] GET /identity/v3/groups/799b2f0fc52e4bfa99efa19842fb0852 => generated 229 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:49.616421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9244a7-fa03-4d98-b844-a299c45d6fc2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:49.620508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-39e62cc9-8104-4b91-819d-bb9d5933ebd5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:49.620508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:49.620508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:49.620508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:49.620508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:49.620508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:49.620508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:49.620508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:49.620508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:49.620508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:49.620508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:49.620508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:49.620508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:49.620508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:49.620508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:49.620508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:49.621389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:49.621389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:49.621389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:49.621389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:49.621389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:49.621389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:49.621389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:49.621389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:49.621389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:49.621389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:49.621389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:49.621389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:49.621389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:49.621389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:49.621389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:49.621389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:49.622250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:49.622250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:49.622250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:49.622250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:49.622250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:49.622250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:49.622250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:49.622250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:49.622250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:49.622250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:49.622250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:49.622250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:49.626424 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 194/416] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:28:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:49.631962 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e62cc9-8104-4b91-819d-bb9d5933ebd5 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:49.762562 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9244a7-fa03-4d98-b844-a299c45d6fc2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:49.818845 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e62cc9-8104-4b91-819d-bb9d5933ebd5 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:49.824982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc99173-82f5-41e1-9c14-977bf1ace0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:49.828905 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc99173-82f5-41e1-9c14-977bf1ace0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:49.829425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc99173-82f5-41e1-9c14-977bf1ace0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:49.829883 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc99173-82f5-41e1-9c14-977bf1ace0f5 None admin] PATH_INFO: `/v3/projects/8e45bcca33094251b8a3ed60293c2eb5` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:49.833318 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc99173-82f5-41e1-9c14-977bf1ace0f5 None admin] RBAC: Authorizing `identity:get_project(project_id=8e45bcca33094251b8a3ed60293c2eb5)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:49.835015 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc99173-82f5-41e1-9c14-977bf1ace0f5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:49.849172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 223/417] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:49 2022] GET /identity/v3/projects/8e45bcca33094251b8a3ed60293c2eb5 => generated 344 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:49.857338 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc99173-82f5-41e1-9c14-977bf1ace0f5 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:49.900172 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b700e-231b-4ba7-94ef-f04d2b9cbea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:49.900843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b700e-231b-4ba7-94ef-f04d2b9cbea0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:49.901123 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b700e-231b-4ba7-94ef-f04d2b9cbea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:49.901639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b700e-231b-4ba7-94ef-f04d2b9cbea0 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:49.903566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b700e-231b-4ba7-94ef-f04d2b9cbea0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:49.905714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b700e-231b-4ba7-94ef-f04d2b9cbea0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:49.907424 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b700e-231b-4ba7-94ef-f04d2b9cbea0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:49.918334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:49.921355 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 195/418] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:28:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:49.929294 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b700e-231b-4ba7-94ef-f04d2b9cbea0 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:49.948303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:50.054269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc99173-82f5-41e1-9c14-977bf1ace0f5 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:50.104667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b700e-231b-4ba7-94ef-f04d2b9cbea0 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:50.118635 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c04235-8b18-441b-b38b-0945deff35f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:50.120447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c04235-8b18-441b-b38b-0945deff35f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:50.120882 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c04235-8b18-441b-b38b-0945deff35f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:50.121571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c04235-8b18-441b-b38b-0945deff35f4 None admin] PATH_INFO: `/v3/projects/8e45bcca33094251b8a3ed60293c2eb5/groups/799b2f0fc52e4bfa99efa19842fb0852/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:50.124892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c04235-8b18-441b-b38b-0945deff35f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e45bcca33094251b8a3ed60293c2eb5, group_id=799b2f0fc52e4bfa99efa19842fb0852, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:50.126547 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c04235-8b18-441b-b38b-0945deff35f4 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:50.144935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a13a9-0937-463a-b00b-a1207c55a456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:50.145705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a13a9-0937-463a-b00b-a1207c55a456 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:50.146014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a13a9-0937-463a-b00b-a1207c55a456 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:50.146294 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a13a9-0937-463a-b00b-a1207c55a456 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:50.146971 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 224/419] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Jul 12 13:28:49 2022] PUT /identity/v3/projects/8e45bcca33094251b8a3ed60293c2eb5/groups/799b2f0fc52e4bfa99efa19842fb0852/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:50.148589 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1dad9e-bd9f-4ddc-b81a-e08001ce8411 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:50.149098 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1dad9e-bd9f-4ddc-b81a-e08001ce8411 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:50.149926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1dad9e-bd9f-4ddc-b81a-e08001ce8411 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:50.153019 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 225/420] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:50.159397 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ec732-e4d4-44aa-b23a-e050f93b70cf None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:50.159729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ec732-e4d4-44aa-b23a-e050f93b70cf None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:50.159960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ec732-e4d4-44aa-b23a-e050f93b70cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:50.183883 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-664ec732-e4d4-44aa-b23a-e050f93b70cf None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:50.184820 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-664ec732-e4d4-44aa-b23a-e050f93b70cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:50.251052 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:50.256346 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a13a9-0937-463a-b00b-a1207c55a456 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:50.257790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a13a9-0937-463a-b00b-a1207c55a456 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:50.267579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 226/421] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:50.290645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ec732-e4d4-44aa-b23a-e050f93b70cf None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:50.325254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:50.369411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 196/422] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:50.379058 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a13a9-0937-463a-b00b-a1207c55a456 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:50.406705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ec732-e4d4-44aa-b23a-e050f93b70cf None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:50.445363 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619dbfd3-4d0c-45aa-ae10-8c3d60a0f998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:50.446614 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619dbfd3-4d0c-45aa-ae10-8c3d60a0f998 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:50.447227 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619dbfd3-4d0c-45aa-ae10-8c3d60a0f998 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:50.447792 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619dbfd3-4d0c-45aa-ae10-8c3d60a0f998 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:50.460140 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619dbfd3-4d0c-45aa-ae10-8c3d60a0f998 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:50.468914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-619dbfd3-4d0c-45aa-ae10-8c3d60a0f998 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:50.468914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:50.468914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:50.468914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:50.468914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:50.468914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:50.468914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:50.468914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:50.468914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:50.468914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:50.468914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:50.468914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:50.468914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:50.468914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:50.468914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:50.468914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:50.469810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:50.469810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:50.469810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:50.469810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:50.469810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:50.469810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:50.469810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:50.469810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:50.469810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:50.469810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:50.469810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:50.469810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:50.469810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:50.469810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:50.469810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:50.469810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:50.471819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:50.471819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:50.471819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:50.471819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:50.471819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:50.471819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:50.471819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:50.471819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:50.471819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:50.471819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:50.471819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:50.471819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:50.474477 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 227/423] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:28:50.481411 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619dbfd3-4d0c-45aa-ae10-8c3d60a0f998 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:50.503621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:50.506883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a13a9-0937-463a-b00b-a1207c55a456 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:50.584973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50957b-5a93-4cb6-8253-4265cbcb447f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:50.586105 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50957b-5a93-4cb6-8253-4265cbcb447f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:50.588251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50957b-5a93-4cb6-8253-4265cbcb447f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:50.588678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50957b-5a93-4cb6-8253-4265cbcb447f None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:50.591240 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50957b-5a93-4cb6-8253-4265cbcb447f None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:50.592874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50957b-5a93-4cb6-8253-4265cbcb447f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:50.597687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 197/424] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:50 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:50.601458 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50957b-5a93-4cb6-8253-4265cbcb447f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:50.620012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:50.645266 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619dbfd3-4d0c-45aa-ae10-8c3d60a0f998 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:50.696296 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd7f52-05ca-428e-b1fc-d47d5d64b60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:50.697125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd7f52-05ca-428e-b1fc-d47d5d64b60b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:50.697678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd7f52-05ca-428e-b1fc-d47d5d64b60b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:50.698104 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd7f52-05ca-428e-b1fc-d47d5d64b60b None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:50.699765 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd7f52-05ca-428e-b1fc-d47d5d64b60b None admin] RBAC: Adding query filter params (name=member) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:50.700184 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd7f52-05ca-428e-b1fc-d47d5d64b60b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:50.701743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd7f52-05ca-428e-b1fc-d47d5d64b60b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:50.713385 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 228/425] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:50.718115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb1244-838a-4c93-838d-2b4c7262af6f None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:50.718115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb1244-838a-4c93-838d-2b4c7262af6f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:50.718115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb1244-838a-4c93-838d-2b4c7262af6f None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:50.721547 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 229/426] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:50.722426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb1244-838a-4c93-838d-2b4c7262af6f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:50.739813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50957b-5a93-4cb6-8253-4265cbcb447f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:50.776738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70431c9-9c5c-406c-9118-8114a54170df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:50.778060 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70431c9-9c5c-406c-9118-8114a54170df None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:50.778558 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70431c9-9c5c-406c-9118-8114a54170df None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:50.779741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70431c9-9c5c-406c-9118-8114a54170df None admin] PATH_INFO: `/v3/groups/9758043b2c3a4673b089931b98c328e4` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:50.781182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70431c9-9c5c-406c-9118-8114a54170df None admin] RBAC: Authorizing `identity:get_group(group_id=9758043b2c3a4673b089931b98c328e4)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:50.782898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70431c9-9c5c-406c-9118-8114a54170df None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:50.785061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 198/427] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jul 12 13:28:50 2022] GET /identity/v3/groups/9758043b2c3a4673b089931b98c328e4 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:50.786545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf1ae2-c5b6-444c-a5a3-291c86a8ea83 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:50.787081 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf1ae2-c5b6-444c-a5a3-291c86a8ea83 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:50.787549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf1ae2-c5b6-444c-a5a3-291c86a8ea83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:50.808295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a4cf1ae2-c5b6-444c-a5a3-291c86a8ea83 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:50.810152 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a4cf1ae2-c5b6-444c-a5a3-291c86a8ea83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:50.846620 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:50.854949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb1244-838a-4c93-838d-2b4c7262af6f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:50.894770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d41ac-cb5b-4e23-b236-4a1179a2b60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:50.895962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d41ac-cb5b-4e23-b236-4a1179a2b60d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:50.896466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d41ac-cb5b-4e23-b236-4a1179a2b60d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:50.896960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d41ac-cb5b-4e23-b236-4a1179a2b60d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:50.902040 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 199/428] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:50.903171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf1ae2-c5b6-444c-a5a3-291c86a8ea83 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:50.939054 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:50.984356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d41ac-cb5b-4e23-b236-4a1179a2b60d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:50.985876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d41ac-cb5b-4e23-b236-4a1179a2b60d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:51.029623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf1ae2-c5b6-444c-a5a3-291c86a8ea83 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:51.068224 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d47b7-8c09-411a-8576-933ec1390732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:51.069318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d47b7-8c09-411a-8576-933ec1390732 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:51.069822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d47b7-8c09-411a-8576-933ec1390732 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:51.070324 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d47b7-8c09-411a-8576-933ec1390732 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:51.071338 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d47b7-8c09-411a-8576-933ec1390732 None admin] RBAC: Adding query filter params (group.id=9758043b2c3a4673b089931b98c328e4, role.id=dbe30950eded4c9ab2bcc0f7545b9bae, scope.project.id=abaed0828ac34adb801becefa86d65b3) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:51.071912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d47b7-8c09-411a-8576-933ec1390732 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:51.073667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d47b7-8c09-411a-8576-933ec1390732 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:51.081127 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 200/429] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jul 12 13:28:50 2022] GET /identity/v3/role_assignments?group.id=9758043b2c3a4673b089931b98c328e4&scope.project.id=abaed0828ac34adb801becefa86d65b3&role.id=dbe30950eded4c9ab2bcc0f7545b9bae => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:51.082384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d47b7-8c09-411a-8576-933ec1390732 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:51.093871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 230/430] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:51.095223 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d41ac-cb5b-4e23-b236-4a1179a2b60d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:51.124594 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:51.180452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:51.236276 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d47b7-8c09-411a-8576-933ec1390732 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:51.236751 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d41ac-cb5b-4e23-b236-4a1179a2b60d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:51.275453 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb40c8a-a8e8-4ef4-9471-3a990d12c680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:51.276640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb40c8a-a8e8-4ef4-9471-3a990d12c680 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:51.277124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb40c8a-a8e8-4ef4-9471-3a990d12c680 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:51.277621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb40c8a-a8e8-4ef4-9471-3a990d12c680 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:51.290072 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb40c8a-a8e8-4ef4-9471-3a990d12c680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:51.297714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54532b76-ee8b-4ebb-bc5a-19432369a32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:51.298628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54532b76-ee8b-4ebb-bc5a-19432369a32a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:51.298914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54532b76-ee8b-4ebb-bc5a-19432369a32a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:51.299191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54532b76-ee8b-4ebb-bc5a-19432369a32a None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:51.300435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54532b76-ee8b-4ebb-bc5a-19432369a32a None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:51.301096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-cbb40c8a-a8e8-4ef4-9471-3a990d12c680 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:51.301096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:51.301096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:51.301096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:51.301096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:51.301096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:51.301096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:51.301096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:51.301096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:51.301096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:51.301096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:51.301096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:51.301096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:51.301096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:51.301096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:51.301096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:51.301946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:51.301946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:51.301946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:51.301946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:51.301946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:51.301946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:51.301946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:51.301946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:51.301946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:51.301946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:51.301946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:51.301946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:51.301946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:51.301946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:51.301946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:51.301946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:51.302764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:51.302764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:51.302764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:51.302764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:51.302764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:51.302764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:51.302764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:51.302764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:51.302764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:51.302764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:51.302764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:28:51.302764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:51.303364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54532b76-ee8b-4ebb-bc5a-19432369a32a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:51.308538 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 231/431] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:51 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:51.308713 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 201/432] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:51.313839 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54532b76-ee8b-4ebb-bc5a-19432369a32a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:51.315162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb40c8a-a8e8-4ef4-9471-3a990d12c680 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:51.455466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54532b76-ee8b-4ebb-bc5a-19432369a32a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:51.478448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:51.482809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb40c8a-a8e8-4ef4-9471-3a990d12c680 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:51.501611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:51.522661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff83242-2b64-4d28-9b94-e8ca57e06d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:51.524163 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff83242-2b64-4d28-9b94-e8ca57e06d92 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:51.524580 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff83242-2b64-4d28-9b94-e8ca57e06d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:51.524921 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff83242-2b64-4d28-9b94-e8ca57e06d92 None admin] PATH_INFO: `/v3/groups/9758043b2c3a4673b089931b98c328e4` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:51.526539 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff83242-2b64-4d28-9b94-e8ca57e06d92 None admin] RBAC: Authorizing `identity:get_group(group_id=9758043b2c3a4673b089931b98c328e4)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:51.528193 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff83242-2b64-4d28-9b94-e8ca57e06d92 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:51.528522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e999267-8613-4a5c-a188-5f0e14c64bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:51.529318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e999267-8613-4a5c-a188-5f0e14c64bfe None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:51.529665 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e999267-8613-4a5c-a188-5f0e14c64bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:51.530146 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 232/433] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jul 12 13:28:51 2022] GET /identity/v3/groups/9758043b2c3a4673b089931b98c328e4 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:51.530259 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e999267-8613-4a5c-a188-5f0e14c64bfe None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:51.531118 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e999267-8613-4a5c-a188-5f0e14c64bfe None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:51.532478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e999267-8613-4a5c-a188-5f0e14c64bfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:51.533877 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e999267-8613-4a5c-a188-5f0e14c64bfe None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:51.536418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff83242-2b64-4d28-9b94-e8ca57e06d92 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:51.552881 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 202/434] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:28:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:51.554505 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e999267-8613-4a5c-a188-5f0e14c64bfe None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:51.684384 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff83242-2b64-4d28-9b94-e8ca57e06d92 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:51.699542 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e999267-8613-4a5c-a188-5f0e14c64bfe None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:51.726150 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a845f-0eb0-4b97-92c1-3cbb35dd5f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:51.726723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a845f-0eb0-4b97-92c1-3cbb35dd5f2f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:51.727010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a845f-0eb0-4b97-92c1-3cbb35dd5f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:51.727281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a845f-0eb0-4b97-92c1-3cbb35dd5f2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:51.728093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a845f-0eb0-4b97-92c1-3cbb35dd5f2f None admin] RBAC: Adding query filter params (group.id=9758043b2c3a4673b089931b98c328e4, role.id=ed842da869724411a5b355cd12f1c5c8, scope.project.id=ae719f7141f34603811594a41497e87c) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:51.728400 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a845f-0eb0-4b97-92c1-3cbb35dd5f2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:51.729834 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a845f-0eb0-4b97-92c1-3cbb35dd5f2f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:51.735666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 233/435] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jul 12 13:28:51 2022] GET /identity/v3/role_assignments?group.id=9758043b2c3a4673b089931b98c328e4&scope.project.id=ae719f7141f34603811594a41497e87c&role.id=ed842da869724411a5b355cd12f1c5c8 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:51.736150 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914c3f2-1531-4a96-8361-bfc5a59b26fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:51.736805 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914c3f2-1531-4a96-8361-bfc5a59b26fe None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:51.736906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84d956-4eec-4b75-a2b4-f2f690b30eb5 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:51.737077 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914c3f2-1531-4a96-8361-bfc5a59b26fe None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:51.737187 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84d956-4eec-4b75-a2b4-f2f690b30eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:51.737362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914c3f2-1531-4a96-8361-bfc5a59b26fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:51.737462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84d956-4eec-4b75-a2b4-f2f690b30eb5 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:51.738849 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 234/436] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:51.744333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28982e-7731-4c78-9fff-60fe637416d9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:51.744633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28982e-7731-4c78-9fff-60fe637416d9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:51.744896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28982e-7731-4c78-9fff-60fe637416d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:51.768748 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8d28982e-7731-4c78-9fff-60fe637416d9 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:51.770020 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8d28982e-7731-4c78-9fff-60fe637416d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:51.772404 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:51.828588 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:51.846259 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914c3f2-1531-4a96-8361-bfc5a59b26fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:51.848704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914c3f2-1531-4a96-8361-bfc5a59b26fe None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:51.858473 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 235/437] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:51.882307 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28982e-7731-4c78-9fff-60fe637416d9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:51.980038 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 203/438] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:28:51.986400 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914c3f2-1531-4a96-8361-bfc5a59b26fe None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:52.011068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28982e-7731-4c78-9fff-60fe637416d9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:52.041102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:52.051412 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778727e-626f-4244-83eb-1a55dbbe372e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:52.052230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778727e-626f-4244-83eb-1a55dbbe372e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:52.052578 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778727e-626f-4244-83eb-1a55dbbe372e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:52.052849 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778727e-626f-4244-83eb-1a55dbbe372e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:52.065094 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778727e-626f-4244-83eb-1a55dbbe372e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:52.073303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-0778727e-626f-4244-83eb-1a55dbbe372e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:52.073303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:52.073303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:52.073303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:52.073303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:52.073303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:52.073303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:52.073303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:52.073303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:52.073303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:52.073303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:52.073303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:52.073303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:52.073303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:52.073303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:52.073303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:52.074407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:52.074407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:52.074407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:52.074407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:52.074407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:52.074407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:52.074407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:52.074407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:52.074407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:52.074407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:52.074407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:52.074407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:52.074407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:52.074407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:52.074407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:52.074407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:52.075229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:52.075229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:52.075229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:52.075229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:52.075229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:52.075229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:52.075229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:52.075229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:52.075229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:52.075229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:52.075229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:52.075229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:52.075229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 236/439] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:28:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:52.080165 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778727e-626f-4244-83eb-1a55dbbe372e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:52.094284 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914c3f2-1531-4a96-8361-bfc5a59b26fe None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:52.128770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:52.129930 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4f12d-5ce2-45f4-8486-2f1f40235479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:52.130839 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4f12d-5ce2-45f4-8486-2f1f40235479 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:52.131185 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4f12d-5ce2-45f4-8486-2f1f40235479 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:52.131423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4f12d-5ce2-45f4-8486-2f1f40235479 None admin] PATH_INFO: `/v3/projects/abaed0828ac34adb801becefa86d65b3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:52.132709 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4f12d-5ce2-45f4-8486-2f1f40235479 None admin] RBAC: Authorizing `identity:get_project(project_id=abaed0828ac34adb801becefa86d65b3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:52.134175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4f12d-5ce2-45f4-8486-2f1f40235479 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:52.136059 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 204/440] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:51 2022] GET /identity/v3/projects/abaed0828ac34adb801becefa86d65b3 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:52.141416 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4f12d-5ce2-45f4-8486-2f1f40235479 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:52.188582 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778727e-626f-4244-83eb-1a55dbbe372e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:52.224867 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1b311-5d7e-43b9-a1d5-9dbc59e7ccf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:52.225539 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1b311-5d7e-43b9-a1d5-9dbc59e7ccf0 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:52.225816 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1b311-5d7e-43b9-a1d5-9dbc59e7ccf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:52.226133 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1b311-5d7e-43b9-a1d5-9dbc59e7ccf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:52.226871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1b311-5d7e-43b9-a1d5-9dbc59e7ccf0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:52.227175 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1b311-5d7e-43b9-a1d5-9dbc59e7ccf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:52.228552 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1b311-5d7e-43b9-a1d5-9dbc59e7ccf0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:52.237479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 237/441] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:28:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:52.242826 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1b311-5d7e-43b9-a1d5-9dbc59e7ccf0 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:52.249756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4f12d-5ce2-45f4-8486-2f1f40235479 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:52.272470 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:52.288798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201693f1-982d-47e3-aed6-cadb2136b359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:52.289649 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201693f1-982d-47e3-aed6-cadb2136b359 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:52.289936 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201693f1-982d-47e3-aed6-cadb2136b359 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:52.290251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201693f1-982d-47e3-aed6-cadb2136b359 None admin] PATH_INFO: `/v3/groups/9758043b2c3a4673b089931b98c328e4` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:52.291399 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201693f1-982d-47e3-aed6-cadb2136b359 None admin] RBAC: Authorizing `identity:get_group(group_id=9758043b2c3a4673b089931b98c328e4)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:52.292909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201693f1-982d-47e3-aed6-cadb2136b359 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:52.299901 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 205/442] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jul 12 13:28:52 2022] GET /identity/v3/groups/9758043b2c3a4673b089931b98c328e4 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:52.301194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201693f1-982d-47e3-aed6-cadb2136b359 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:52.347982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1b311-5d7e-43b9-a1d5-9dbc59e7ccf0 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:52.356565 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:52.386603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81185160-b28b-44fd-9673-1304bd57e2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:52.387253 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81185160-b28b-44fd-9673-1304bd57e2db None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:52.387539 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81185160-b28b-44fd-9673-1304bd57e2db None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:52.387813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81185160-b28b-44fd-9673-1304bd57e2db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:52.412590 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201693f1-982d-47e3-aed6-cadb2136b359 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:52.450228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6456f29-33f7-44e2-9c71-cf2264debdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:52.451107 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6456f29-33f7-44e2-9c71-cf2264debdd9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:52.451696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6456f29-33f7-44e2-9c71-cf2264debdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:52.452117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6456f29-33f7-44e2-9c71-cf2264debdd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:52.453080 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6456f29-33f7-44e2-9c71-cf2264debdd9 None admin] RBAC: Adding query filter params (group.id=9758043b2c3a4673b089931b98c328e4, role.id=ed842da869724411a5b355cd12f1c5c8, scope.project.id=abaed0828ac34adb801becefa86d65b3) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:52.453398 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6456f29-33f7-44e2-9c71-cf2264debdd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:52.455039 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6456f29-33f7-44e2-9c71-cf2264debdd9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:52.457776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81185160-b28b-44fd-9673-1304bd57e2db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:52.459295 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81185160-b28b-44fd-9673-1304bd57e2db None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:52.467766 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 206/443] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jul 12 13:28:52 2022] GET /identity/v3/role_assignments?group.id=9758043b2c3a4673b089931b98c328e4&scope.project.id=abaed0828ac34adb801becefa86d65b3&role.id=ed842da869724411a5b355cd12f1c5c8 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:52.500571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afc084-5049-47d3-9092-66fcea1e4bde None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:52.501590 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afc084-5049-47d3-9092-66fcea1e4bde None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:52.501891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afc084-5049-47d3-9092-66fcea1e4bde None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:52.503814 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 207/444] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:28:52.509179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cdb91-9847-4570-b3dd-622675155008 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:52.509468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cdb91-9847-4570-b3dd-622675155008 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:52.509746 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cdb91-9847-4570-b3dd-622675155008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:52.527877 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-362cdb91-9847-4570-b3dd-622675155008 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:52.529776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-362cdb91-9847-4570-b3dd-622675155008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:52.583686 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 238/445] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 13:28:52.589466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:52.590246 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81185160-b28b-44fd-9673-1304bd57e2db None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:52.599269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:52.618056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 208/446] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:52.637850 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362cdb91-9847-4570-b3dd-622675155008 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:52.707612 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81185160-b28b-44fd-9673-1304bd57e2db None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:52.750106 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29e4e6-249a-4f7c-a6a5-ebd49ecee215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:52.751145 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29e4e6-249a-4f7c-a6a5-ebd49ecee215 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:52.751373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29e4e6-249a-4f7c-a6a5-ebd49ecee215 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:52.751647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29e4e6-249a-4f7c-a6a5-ebd49ecee215 None admin] PATH_INFO: `/v3/projects/ae719f7141f34603811594a41497e87c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:52.753145 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29e4e6-249a-4f7c-a6a5-ebd49ecee215 None admin] RBAC: Authorizing `identity:get_project(project_id=ae719f7141f34603811594a41497e87c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:52.754650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29e4e6-249a-4f7c-a6a5-ebd49ecee215 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:52.756750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 239/447] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:52 2022] GET /identity/v3/projects/ae719f7141f34603811594a41497e87c => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:52.764223 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29e4e6-249a-4f7c-a6a5-ebd49ecee215 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:52.771382 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362cdb91-9847-4570-b3dd-622675155008 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:52.812412 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46646e-a682-4c7d-897a-ff4ac6b3b75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:52.813378 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46646e-a682-4c7d-897a-ff4ac6b3b75e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:52.813754 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46646e-a682-4c7d-897a-ff4ac6b3b75e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:52.814190 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46646e-a682-4c7d-897a-ff4ac6b3b75e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:52.828813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46646e-a682-4c7d-897a-ff4ac6b3b75e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:52.837526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-5e46646e-a682-4c7d-897a-ff4ac6b3b75e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:52.837526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:52.837526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:52.837526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:52.837526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:52.837526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:52.837526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:52.837526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:52.837526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:52.837526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:52.837526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:52.837526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:52.837526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:52.837526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:52.837526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:52.837526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:52.838995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:52.838995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:52.838995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:52.838995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:52.838995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:52.838995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:52.838995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:52.838995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:52.838995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:52.838995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:52.838995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:52.838995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:52.838995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:52.838995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:52.838995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:52.838995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:52.839787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:52.839787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:52.839787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:52.839787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:52.839787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:52.839787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:52.839787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:52.839787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:52.839787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:52.839787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:52.839787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:28:52.839787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:52.839787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 209/448] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:28:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:28:52.845070 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46646e-a682-4c7d-897a-ff4ac6b3b75e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:52.869127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:52.901635 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29e4e6-249a-4f7c-a6a5-ebd49ecee215 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:52.905601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:52.953113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fb902-57ca-48ae-adbc-232fbd0b45ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:52.954055 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fb902-57ca-48ae-adbc-232fbd0b45ea None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:52.954344 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fb902-57ca-48ae-adbc-232fbd0b45ea None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:52.955135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fb902-57ca-48ae-adbc-232fbd0b45ea None admin] PATH_INFO: `/v3/groups/9758043b2c3a4673b089931b98c328e4` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:52.956412 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0fb902-57ca-48ae-adbc-232fbd0b45ea None admin] RBAC: Authorizing `identity:get_group(group_id=9758043b2c3a4673b089931b98c328e4)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:52.958543 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0fb902-57ca-48ae-adbc-232fbd0b45ea None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:52.960203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 240/449] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jul 12 13:28:52 2022] GET /identity/v3/groups/9758043b2c3a4673b089931b98c328e4 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:52.966542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fb902-57ca-48ae-adbc-232fbd0b45ea None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:52.984478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46646e-a682-4c7d-897a-ff4ac6b3b75e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:53.026212 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214d842-1f0d-4bc9-bea2-a8c419e93769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:53.026810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214d842-1f0d-4bc9-bea2-a8c419e93769 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:53.027097 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214d842-1f0d-4bc9-bea2-a8c419e93769 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:53.027368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214d842-1f0d-4bc9-bea2-a8c419e93769 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:53.028118 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214d842-1f0d-4bc9-bea2-a8c419e93769 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:53.028425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214d842-1f0d-4bc9-bea2-a8c419e93769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:53.029889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214d842-1f0d-4bc9-bea2-a8c419e93769 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:53.039390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 210/450] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:28:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:53.045110 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214d842-1f0d-4bc9-bea2-a8c419e93769 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:53.087058 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fb902-57ca-48ae-adbc-232fbd0b45ea None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:53.129247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb5c1f-0b45-4f89-beca-3d72f5f8a27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:53.129919 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb5c1f-0b45-4f89-beca-3d72f5f8a27e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:53.130235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb5c1f-0b45-4f89-beca-3d72f5f8a27e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:53.130977 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb5c1f-0b45-4f89-beca-3d72f5f8a27e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:53.131833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb5c1f-0b45-4f89-beca-3d72f5f8a27e None admin] RBAC: Adding query filter params (group.id=9758043b2c3a4673b089931b98c328e4, role.id=dbe30950eded4c9ab2bcc0f7545b9bae, scope.project.id=ae719f7141f34603811594a41497e87c) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:53.132154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb5c1f-0b45-4f89-beca-3d72f5f8a27e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:53.134034 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb5c1f-0b45-4f89-beca-3d72f5f8a27e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:53.141094 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 241/451] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jul 12 13:28:52 2022] GET /identity/v3/role_assignments?group.id=9758043b2c3a4673b089931b98c328e4&scope.project.id=ae719f7141f34603811594a41497e87c&role.id=dbe30950eded4c9ab2bcc0f7545b9bae => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:53.162869 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214d842-1f0d-4bc9-bea2-a8c419e93769 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:53.166385 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:53.204269 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b9cf9-a6b0-40df-90a5-341a8d1f38c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:53.204937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b9cf9-a6b0-40df-90a5-341a8d1f38c2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:53.205223 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b9cf9-a6b0-40df-90a5-341a8d1f38c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:53.205507 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b9cf9-a6b0-40df-90a5-341a8d1f38c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:53.277329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b9cf9-a6b0-40df-90a5-341a8d1f38c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:53.278747 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b9cf9-a6b0-40df-90a5-341a8d1f38c2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:53.377940 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 211/452] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:28:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 13:28:53.383555 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb5c1f-0b45-4f89-beca-3d72f5f8a27e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:53.392596 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:53.493381 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb5c1f-0b45-4f89-beca-3d72f5f8a27e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:53.526367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cf4ca-f262-460f-ac3d-1090184a073c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:53.527220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cf4ca-f262-460f-ac3d-1090184a073c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:53.527660 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cf4ca-f262-460f-ac3d-1090184a073c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:53.527765 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cf4ca-f262-460f-ac3d-1090184a073c None admin] PATH_INFO: `/v3/projects/8e45bcca33094251b8a3ed60293c2eb5` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:53.528922 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cf4ca-f262-460f-ac3d-1090184a073c None admin] RBAC: Authorizing `identity:get_project(project_id=8e45bcca33094251b8a3ed60293c2eb5)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:53.530406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cf4ca-f262-460f-ac3d-1090184a073c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:53.532070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 242/453] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:28:53 2022] GET /identity/v3/projects/8e45bcca33094251b8a3ed60293c2eb5 => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:53.537506 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b9cf9-a6b0-40df-90a5-341a8d1f38c2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:53.540552 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:53.635224 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b9cf9-a6b0-40df-90a5-341a8d1f38c2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:53.668011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6957ce-f687-442e-abf7-a14e93273fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:53.668859 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6957ce-f687-442e-abf7-a14e93273fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:53.669141 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6957ce-f687-442e-abf7-a14e93273fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:53.669412 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6957ce-f687-442e-abf7-a14e93273fc6 None admin] PATH_INFO: `/v3/groups/799b2f0fc52e4bfa99efa19842fb0852` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:53.671166 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6957ce-f687-442e-abf7-a14e93273fc6 None admin] RBAC: Authorizing `identity:get_group(group_id=799b2f0fc52e4bfa99efa19842fb0852)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:53.672631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6957ce-f687-442e-abf7-a14e93273fc6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:53.674318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 212/454] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jul 12 13:28:53 2022] GET /identity/v3/groups/799b2f0fc52e4bfa99efa19842fb0852 => generated 229 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:53.680117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cf4ca-f262-460f-ac3d-1090184a073c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:53.754356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:53.779553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cf4ca-f262-460f-ac3d-1090184a073c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:53.812589 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ccd6e-60b4-4be8-9a5c-f44f5fb1056c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:53.813236 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ccd6e-60b4-4be8-9a5c-f44f5fb1056c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:53.813521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ccd6e-60b4-4be8-9a5c-f44f5fb1056c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:53.813798 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ccd6e-60b4-4be8-9a5c-f44f5fb1056c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:53.814650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ccd6e-60b4-4be8-9a5c-f44f5fb1056c None admin] RBAC: Adding query filter params (group.id=799b2f0fc52e4bfa99efa19842fb0852, role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=8e45bcca33094251b8a3ed60293c2eb5) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:53.814966 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ccd6e-60b4-4be8-9a5c-f44f5fb1056c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:53.816412 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ccd6e-60b4-4be8-9a5c-f44f5fb1056c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:53.822151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 243/455] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jul 12 13:28:53 2022] GET /identity/v3/role_assignments?group.id=799b2f0fc52e4bfa99efa19842fb0852&scope.project.id=8e45bcca33094251b8a3ed60293c2eb5&role.id=864a6966630544e8bd68bcafba905c6f => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:56.221767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cf792-1287-4dca-b095-288460a925b5 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:56.222336 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cf792-1287-4dca-b095-288460a925b5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:56.222336 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cf792-1287-4dca-b095-288460a925b5 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:56.223946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 213/456] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:56.229430 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a0269-6905-42da-86c7-23e0fc994107 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:56.230107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a0269-6905-42da-86c7-23e0fc994107 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:56.230107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a0269-6905-42da-86c7-23e0fc994107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:56.254990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-908a0269-6905-42da-86c7-23e0fc994107 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:56.256184 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-908a0269-6905-42da-86c7-23e0fc994107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:56.344258 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 244/457] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:28:56.381033 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cf792-1287-4dca-b095-288460a925b5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:56.469121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:56.511307 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cf792-1287-4dca-b095-288460a925b5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:56.554037 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8697420-db3a-4854-b046-32a172732dab None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:56.554374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8697420-db3a-4854-b046-32a172732dab None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:56.554585 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8697420-db3a-4854-b046-32a172732dab None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:56.555431 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a0b0e-bee8-4e70-a137-89e17e6afbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:56.556229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a0b0e-bee8-4e70-a137-89e17e6afbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:56.556507 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a0b0e-bee8-4e70-a137-89e17e6afbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:56.556775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a0b0e-bee8-4e70-a137-89e17e6afbf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:56.557155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 245/458] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:28:56.562113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611a167-a63d-46b9-a22d-f77534fd7ce2 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:56.562758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611a167-a63d-46b9-a22d-f77534fd7ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:56.563240 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611a167-a63d-46b9-a22d-f77534fd7ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:56.567915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a0b0e-bee8-4e70-a137-89e17e6afbf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:56.570056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a0b0e-bee8-4e70-a137-89e17e6afbf3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:56.590756 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8611a167-a63d-46b9-a22d-f77534fd7ce2 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:56.600046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-8b4a0b0e-bee8-4e70-a137-89e17e6afbf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:28:56.600046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:56.600046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:56.600046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:56.600046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:56.600046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:56.600046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:56.600046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:56.600046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:56.600046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:56.600046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:56.600046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:56.600046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:28:56.600046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:28:56.600046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:28:56.600935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:28:56.600935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:56.600935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:56.600935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:56.600935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:56.600935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:56.600935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:56.600935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:56.600935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:56.600935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:56.600935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:56.600935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:56.600935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:56.600935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:56.600935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:56.600935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:56.600935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:56.601790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:28:56.601790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:28:56.601790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:28:56.601790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:56.602030 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8611a167-a63d-46b9-a22d-f77534fd7ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:56.604175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 214/459] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:56.610011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a0b0e-bee8-4e70-a137-89e17e6afbf3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:56.703386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:56.711503 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 246/460] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:56.715305 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43166b82-0291-4caa-9c4e-e449b6e67a0f None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:56.715746 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43166b82-0291-4caa-9c4e-e449b6e67a0f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:56.716149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43166b82-0291-4caa-9c4e-e449b6e67a0f None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:56.717714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 247/461] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:56.723006 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80f40c-5732-413c-93ae-bb43b8179b61 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:56.723595 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80f40c-5732-413c-93ae-bb43b8179b61 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:56.723937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80f40c-5732-413c-93ae-bb43b8179b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:56.742194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ce80f40c-5732-413c-93ae-bb43b8179b61 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:56.743388 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ce80f40c-5732-413c-93ae-bb43b8179b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:56.778158 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a0b0e-bee8-4e70-a137-89e17e6afbf3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:56.802275 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:56.814207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe13e6-fc9e-42a4-94d4-377c5a9cc403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:56.814871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe13e6-fc9e-42a4-94d4-377c5a9cc403 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:56.815154 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe13e6-fc9e-42a4-94d4-377c5a9cc403 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:56.815427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe13e6-fc9e-42a4-94d4-377c5a9cc403 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:56.816200 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe13e6-fc9e-42a4-94d4-377c5a9cc403 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:56.816506 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe13e6-fc9e-42a4-94d4-377c5a9cc403 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:56.817951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe13e6-fc9e-42a4-94d4-377c5a9cc403 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:56.820974 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 248/462] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:56.826207 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80f40c-5732-413c-93ae-bb43b8179b61 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:56.831063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 215/463] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:56.836687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe13e6-fc9e-42a4-94d4-377c5a9cc403 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:56.946987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe13e6-fc9e-42a4-94d4-377c5a9cc403 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:56.951713 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80f40c-5732-413c-93ae-bb43b8179b61 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:56.981112 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:56.981646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adaff1d-4ccd-4eb4-a157-a351cf77cd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:56.982572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adaff1d-4ccd-4eb4-a157-a351cf77cd48 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:56.983000 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adaff1d-4ccd-4eb4-a157-a351cf77cd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:56.983449 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adaff1d-4ccd-4eb4-a157-a351cf77cd48 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:56.984296 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adaff1d-4ccd-4eb4-a157-a351cf77cd48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:56.985880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adaff1d-4ccd-4eb4-a157-a351cf77cd48 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:56.990377 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2319823-5a05-4ae4-b421-46324d9bca57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:56.991384 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2319823-5a05-4ae4-b421-46324d9bca57 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:56.991815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2319823-5a05-4ae4-b421-46324d9bca57 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:56.992233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2319823-5a05-4ae4-b421-46324d9bca57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:57.006399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2319823-5a05-4ae4-b421-46324d9bca57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:57.008376 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2319823-5a05-4ae4-b421-46324d9bca57 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:57.015155 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 216/464] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:28:56 2022] POST /identity/v3/users => generated 246 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:57.016387 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adaff1d-4ccd-4eb4-a157-a351cf77cd48 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:57.034650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-d2319823-5a05-4ae4-b421-46324d9bca57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:28:57.034650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:57.034650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:57.034650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:57.034650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:57.034650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:57.034650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:57.034650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:57.034650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:57.034650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:57.034650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:57.034650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:57.034650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:28:57.034650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:28:57.034650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:28:57.035805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:28:57.035805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:57.035805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:57.035805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:57.035805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:57.035805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:57.035805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:57.035805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:57.035805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:57.035805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:57.035805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:57.035805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:57.035805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:57.035805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:57.035805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:57.035805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:57.035805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:57.036786 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:28:57.036786 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:28:57.036786 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:28:57.036786 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:28:57.039361 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 249/465] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:57.055039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2319823-5a05-4ae4-b421-46324d9bca57 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:57.090131 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:57.152905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adaff1d-4ccd-4eb4-a157-a351cf77cd48 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:57.192395 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6212e-7e31-463c-a7dc-3017ea88d571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:57.193214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6212e-7e31-463c-a7dc-3017ea88d571 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:57.193551 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6212e-7e31-463c-a7dc-3017ea88d571 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:57.193891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6212e-7e31-463c-a7dc-3017ea88d571 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:57.206035 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6212e-7e31-463c-a7dc-3017ea88d571 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:57.206614 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2319823-5a05-4ae4-b421-46324d9bca57 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:57.207765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6212e-7e31-463c-a7dc-3017ea88d571 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:57.227132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-12e6212e-7e31-463c-a7dc-3017ea88d571 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:28:57.227132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:57.227132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:57.227132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:57.227132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:57.227132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:57.227132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:57.227132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:57.227132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:57.227132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:57.227132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:57.227132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:57.227132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:28:57.227132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:28:57.227132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:28:57.228664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:28:57.228664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:57.228664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:57.228664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:57.228664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:57.228664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:57.228664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:57.228664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:57.228664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:57.228664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:57.228664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:57.228664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:57.228664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:57.228664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:57.228664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:57.228664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:57.228664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:57.229716 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:28:57.229716 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:28:57.229716 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:28:57.229716 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:57.234369 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 217/466] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:28:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:57.236637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435644f2-9cce-4506-8cc1-ab6c6b41196a None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:57.236928 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435644f2-9cce-4506-8cc1-ab6c6b41196a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:57.237202 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435644f2-9cce-4506-8cc1-ab6c6b41196a None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:57.239444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 218/467] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:57.240325 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435644f2-9cce-4506-8cc1-ab6c6b41196a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:57.252387 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ad7e7-790e-4c83-a2e4-13410bf49cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:57.253651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ad7e7-790e-4c83-a2e4-13410bf49cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:57.254135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ad7e7-790e-4c83-a2e4-13410bf49cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:57.254557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ad7e7-790e-4c83-a2e4-13410bf49cf8 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:57.255444 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ad7e7-790e-4c83-a2e4-13410bf49cf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:57.255897 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ad7e7-790e-4c83-a2e4-13410bf49cf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:57.257461 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ad7e7-790e-4c83-a2e4-13410bf49cf8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:57.276631 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 250/468] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:57.278194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdedcf-37d6-4d91-ac23-d4ca056f5cc1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:57.278487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdedcf-37d6-4d91-ac23-d4ca056f5cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:57.278814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdedcf-37d6-4d91-ac23-d4ca056f5cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:57.296418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-73fdedcf-37d6-4d91-ac23-d4ca056f5cc1 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:57.297553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-73fdedcf-37d6-4d91-ac23-d4ca056f5cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:57.346123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:57.353553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:57.368559 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435644f2-9cce-4506-8cc1-ab6c6b41196a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:57.392891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 251/469] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:57.394158 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fdedcf-37d6-4d91-ac23-d4ca056f5cc1 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:57.431565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b1af0-7131-403d-a376-3af49abcaffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:57.432608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b1af0-7131-403d-a376-3af49abcaffe None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:57.433093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b1af0-7131-403d-a376-3af49abcaffe None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:57.433568 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b1af0-7131-403d-a376-3af49abcaffe None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:57.434678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b1af0-7131-403d-a376-3af49abcaffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:57.435220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b1af0-7131-403d-a376-3af49abcaffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:57.436806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b1af0-7131-403d-a376-3af49abcaffe None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:57.454725 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 219/470] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:57.460120 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b1af0-7131-403d-a376-3af49abcaffe None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:57.509445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fdedcf-37d6-4d91-ac23-d4ca056f5cc1 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:57.547353 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63a407-47a5-4c36-b6fa-06b9aa23ecaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:57.547957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63a407-47a5-4c36-b6fa-06b9aa23ecaa None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:57.548247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63a407-47a5-4c36-b6fa-06b9aa23ecaa None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:57.548745 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63a407-47a5-4c36-b6fa-06b9aa23ecaa None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:57.549233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63a407-47a5-4c36-b6fa-06b9aa23ecaa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:57.550760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63a407-47a5-4c36-b6fa-06b9aa23ecaa None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:57.575875 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b1af0-7131-403d-a376-3af49abcaffe None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:57.581989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 252/471] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:28:57 2022] POST /identity/v3/users => generated 248 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:57.581989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63a407-47a5-4c36-b6fa-06b9aa23ecaa None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:57.651769 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:57.674351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74f589-ff67-4ea3-8c23-0f8667bf2071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:57.675629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74f589-ff67-4ea3-8c23-0f8667bf2071 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:57.675915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74f589-ff67-4ea3-8c23-0f8667bf2071 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:57.676360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74f589-ff67-4ea3-8c23-0f8667bf2071 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:57.691861 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74f589-ff67-4ea3-8c23-0f8667bf2071 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:57.693650 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74f589-ff67-4ea3-8c23-0f8667bf2071 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:57.720512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-7a74f589-ff67-4ea3-8c23-0f8667bf2071 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:28:57.720512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:57.720512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:57.720512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:57.720512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:57.720512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:57.720512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:57.720512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:57.720512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:57.720512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:57.720512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:57.720512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:57.720512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:28:57.720512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:28:57.720512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:28:57.721427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:28:57.721427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:57.721427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:57.721427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:57.721427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:57.721427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:57.721427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:57.721427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:57.721427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:57.721427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:57.721427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:57.721427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:57.721427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:57.721427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:57.721427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:57.721427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:57.721427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:57.722311 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:28:57.722311 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:28:57.722311 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:28:57.722311 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:57.727721 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 220/472] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:28:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:57.729329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ab9d6-bbd8-438e-9b3a-560e244eeb78 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:57.730409 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ab9d6-bbd8-438e-9b3a-560e244eeb78 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:57.730709 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ab9d6-bbd8-438e-9b3a-560e244eeb78 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:57.733567 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 221/473] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:57.734379 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ab9d6-bbd8-438e-9b3a-560e244eeb78 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:57.740565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:57.760603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63a407-47a5-4c36-b6fa-06b9aa23ecaa None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:57.810714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c2611-b0a9-4bd9-86f8-622a9dcd5fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:57.812301 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c2611-b0a9-4bd9-86f8-622a9dcd5fce None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:57.812592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c2611-b0a9-4bd9-86f8-622a9dcd5fce None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:57.812866 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c2611-b0a9-4bd9-86f8-622a9dcd5fce None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:57.813578 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c2611-b0a9-4bd9-86f8-622a9dcd5fce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:57.815093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c2611-b0a9-4bd9-86f8-622a9dcd5fce None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:57.838568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 253/474] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:28:57 2022] POST /identity/v3/users => generated 248 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:57.841842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322a807-b303-4327-b9db-619b108f4c60 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:57.842414 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322a807-b303-4327-b9db-619b108f4c60 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:57.842838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322a807-b303-4327-b9db-619b108f4c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:57.870556 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a322a807-b303-4327-b9db-619b108f4c60 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:57.872207 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a322a807-b303-4327-b9db-619b108f4c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:57.904512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ab9d6-bbd8-438e-9b3a-560e244eeb78 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:57.977122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48512f2e-5b49-40a3-a9e3-2dceee9ee81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:57.978068 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48512f2e-5b49-40a3-a9e3-2dceee9ee81f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:57.978497 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48512f2e-5b49-40a3-a9e3-2dceee9ee81f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:57.978914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48512f2e-5b49-40a3-a9e3-2dceee9ee81f None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:57.979821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48512f2e-5b49-40a3-a9e3-2dceee9ee81f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:57.980274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48512f2e-5b49-40a3-a9e3-2dceee9ee81f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:57.981885 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48512f2e-5b49-40a3-a9e3-2dceee9ee81f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:58.007688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 254/475] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:58.008677 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 222/476] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:58.017115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322a807-b303-4327-b9db-619b108f4c60 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:58.036565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48512f2e-5b49-40a3-a9e3-2dceee9ee81f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:58.041927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:58.125101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:58.167315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322a807-b303-4327-b9db-619b108f4c60 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:58.170272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48512f2e-5b49-40a3-a9e3-2dceee9ee81f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:58.210783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc19298-f9dc-4769-ac54-0980ffb04978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:58.211180 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e949a92-eb10-4626-8f82-4b8c0343506e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:58.211451 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc19298-f9dc-4769-ac54-0980ffb04978 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:58.211732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc19298-f9dc-4769-ac54-0980ffb04978 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:58.211851 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e949a92-eb10-4626-8f82-4b8c0343506e None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:58.212020 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc19298-f9dc-4769-ac54-0980ffb04978 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:58.212123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e949a92-eb10-4626-8f82-4b8c0343506e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:58.212403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e949a92-eb10-4626-8f82-4b8c0343506e None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:58.212764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc19298-f9dc-4769-ac54-0980ffb04978 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:58.213115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e949a92-eb10-4626-8f82-4b8c0343506e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:58.214586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e949a92-eb10-4626-8f82-4b8c0343506e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:58.215386 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc19298-f9dc-4769-ac54-0980ffb04978 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:58.223873 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.sql.core [None req-1bc19298-f9dc-4769-ac54-0980ffb04978 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 13:28:58.223873 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 12 13:28:58.223873 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [parameters: {'id': '0e69496c577d47d6bb453f15a5297b00', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 12 13:28:58.223873 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80500) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 13:28:58.242271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-1bc19298-f9dc-4769-ac54-0980ffb04978 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 12 13:28:58.242271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:58.242271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 13:28:58.242271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 13:28:58.242271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 13:28:58.242271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 13:28:58.242271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 13:28:58.242271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 13:28:58.242271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 13:28:58.242271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application conn.query(q) Jul 12 13:28:58.242271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 13:28:58.242271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 13:28:58.242271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 13:28:58.242271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application result.read() Jul 12 13:28:58.242271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return ref.to_dict() Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 13:28:58.243577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application next(self.gen) Jul 12 13:28:58.245707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 13:28:58.245707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application yield resource Jul 12 13:28:58.245707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 13:28:58.245707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application next(self.gen) Jul 12 13:28:58.245707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 13:28:58.245707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 13:28:58.245707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 13:28:58.245707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 13:28:58.245707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 13:28:58.245707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise self.value Jul 12 13:28:58.245707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 13:28:58.245707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 13:28:58.245707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 13:28:58.245707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application session.commit() Jul 12 13:28:58.245707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 13:28:58.245707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 13:28:58.245707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 13:28:58.248408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 13:28:58.248408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 13:28:58.248408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self.session.flush() Jul 12 13:28:58.248408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 13:28:58.248408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 13:28:58.248408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 13:28:58.248408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 13:28:58.248408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 13:28:58.248408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application compat.raise_( Jul 12 13:28:58.248408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 13:28:58.248408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception Jul 12 13:28:58.248408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 13:28:58.248408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 13:28:58.248408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 13:28:58.248408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 13:28:58.248408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 13:28:58.248408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 13:28:58.252437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 13:28:58.252437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 13:28:58.252437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 12 13:28:58.252437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 12 13:28:58.252437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 13:28:58.252437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 13:28:58.252437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 13:28:58.252437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 13:28:58.252437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 13:28:58.252437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 13:28:58.252437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 13:28:58.252437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 13:28:58.252437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 13:28:58.252437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 13:28:58.252437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 13:28:58.252437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception Jul 12 13:28:58.259115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 13:28:58.259115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 13:28:58.259115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 13:28:58.259115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 13:28:58.259115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 13:28:58.259115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 13:28:58.259115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 13:28:58.259115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application conn.query(q) Jul 12 13:28:58.259115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 13:28:58.259115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 13:28:58.259115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 13:28:58.259115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application result.read() Jul 12 13:28:58.259115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 13:28:58.259115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 13:28:58.259115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 13:28:58.259115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 13:28:58.261130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 13:28:58.261130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 13:28:58.261130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 13:28:58.261130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 13:28:58.261130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 13:28:58.261130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 12 13:28:58.261130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [parameters: {'id': '0e69496c577d47d6bb453f15a5297b00', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 12 13:28:58.261130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 13:28:58.261130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:58.261130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:28:58.261130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:58.261130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:58.261130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:58.261130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:58.261130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:58.261130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:58.263793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:58.263793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:58.263793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:58.263793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:58.263793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:58.263793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:58.263793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 12 13:28:58.263793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 12 13:28:58.263793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:58.263793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:58.263793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 12 13:28:58.263793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 12 13:28:58.263793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 13:28:58.263793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 13:28:58.263793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 12 13:28:58.263793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:58.265829 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 223/477] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Jul 12 13:28:58 2022] POST /identity/v3/roles => generated 144 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:28:58.265988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 255/478] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:28:58 2022] POST /identity/v3/users => generated 249 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:58.266467 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc19298-f9dc-4769-ac54-0980ffb04978 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:58.450055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc19298-f9dc-4769-ac54-0980ffb04978 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:58.477975 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:58.486639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b4098-44c0-4373-89e7-0238a37c3a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:58.488985 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b4098-44c0-4373-89e7-0238a37c3a49 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:58.489268 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b4098-44c0-4373-89e7-0238a37c3a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:58.489541 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b4098-44c0-4373-89e7-0238a37c3a49 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:58.503434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b4098-44c0-4373-89e7-0238a37c3a49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:58.511810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-a05b4098-44c0-4373-89e7-0238a37c3a49 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:58.511810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:58.511810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:58.511810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:58.511810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:58.511810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:58.511810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:58.511810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:58.511810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:58.511810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:58.511810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:58.511810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:58.511810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:58.511810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:58.511810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:58.511810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:58.514621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:58.514621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:58.514621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:58.514621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:58.514621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:58.514621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:58.514621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:58.514621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:58.514621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:58.514621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:58.514621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:58.514621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:58.514621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:58.514621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:58.514621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:58.514621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:58.516637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:58.516637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:58.516637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:58.516637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:58.516637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:58.516637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:58.516637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:58.516637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:58.516637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:58.516637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:58.516637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:28:58.516637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:58.516637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 224/479] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:28:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:58.519317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e949a92-eb10-4626-8f82-4b8c0343506e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:58.610073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:58.665382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e949a92-eb10-4626-8f82-4b8c0343506e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:58.705002 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13251274-4e89-4bbf-8a99-31145b44f1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:58.705589 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13251274-4e89-4bbf-8a99-31145b44f1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:58.705868 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13251274-4e89-4bbf-8a99-31145b44f1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:58.706173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13251274-4e89-4bbf-8a99-31145b44f1d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:58.706913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13251274-4e89-4bbf-8a99-31145b44f1d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:58.707217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13251274-4e89-4bbf-8a99-31145b44f1d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:58.708617 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13251274-4e89-4bbf-8a99-31145b44f1d3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:58.720055 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 256/480] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:28:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:59.466031 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a971861-c040-40e3-9187-a1d9efcca8c5 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:59.466031 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a971861-c040-40e3-9187-a1d9efcca8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:59.466031 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a971861-c040-40e3-9187-a1d9efcca8c5 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:59.468211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 225/481] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:28:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:59.483521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530618b-4212-40a4-8e26-b409e1b5dcec None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:59.484435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530618b-4212-40a4-8e26-b409e1b5dcec None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:59.485066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530618b-4212-40a4-8e26-b409e1b5dcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:59.508353 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4530618b-4212-40a4-8e26-b409e1b5dcec None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:59.509866 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4530618b-4212-40a4-8e26-b409e1b5dcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:59.596562 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 257/482] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:28:59.634586 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a971861-c040-40e3-9187-a1d9efcca8c5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:59.752938 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a971861-c040-40e3-9187-a1d9efcca8c5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:59.788649 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e661893-8be6-44b8-8fac-03b814c5a013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:59.789446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e661893-8be6-44b8-8fac-03b814c5a013 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:59.789727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e661893-8be6-44b8-8fac-03b814c5a013 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:59.790025 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e661893-8be6-44b8-8fac-03b814c5a013 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:59.800953 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e661893-8be6-44b8-8fac-03b814c5a013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:28:59.808732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-1e661893-8be6-44b8-8fac-03b814c5a013 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:28:59.808732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:28:59.808732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:28:59.808732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:28:59.808732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:28:59.808732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:28:59.808732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:28:59.808732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:28:59.808732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:28:59.808732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:28:59.808732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:28:59.808732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:28:59.808732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:28:59.808732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:28:59.808732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:28:59.808732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:28:59.809605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:28:59.809605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:28:59.809605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:28:59.809605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:28:59.809605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:28:59.809605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:28:59.809605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:28:59.809605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:28:59.809605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:28:59.809605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:28:59.809605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:28:59.809605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:28:59.809605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:28:59.809605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:28:59.809605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:28:59.809605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:28:59.810601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:28:59.810601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:28:59.810601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:28:59.810601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:28:59.810601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:28:59.810601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:28:59.810601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:28:59.810601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:28:59.810601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:28:59.810601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:28:59.810601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:28:59.810601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:28:59.810601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 226/483] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:28:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:28:59.815920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530618b-4212-40a4-8e26-b409e1b5dcec None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:59.914339 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:59.970359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530618b-4212-40a4-8e26-b409e1b5dcec None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:00.021355 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c565521-dafe-4b21-aa30-e0607756258a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:00.022535 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c565521-dafe-4b21-aa30-e0607756258a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:00.023042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c565521-dafe-4b21-aa30-e0607756258a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:00.023517 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c565521-dafe-4b21-aa30-e0607756258a None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:00.024457 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c565521-dafe-4b21-aa30-e0607756258a None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:00.024983 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c565521-dafe-4b21-aa30-e0607756258a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:00.026601 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c565521-dafe-4b21-aa30-e0607756258a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:00.037645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 258/484] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:28:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:00.042742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e661893-8be6-44b8-8fac-03b814c5a013 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:00.112468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:00.181428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e661893-8be6-44b8-8fac-03b814c5a013 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:00.215778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c201f-00e3-4c71-a936-c679834b0237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:00.217133 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c201f-00e3-4c71-a936-c679834b0237 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:00.217416 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c201f-00e3-4c71-a936-c679834b0237 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:00.217691 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c201f-00e3-4c71-a936-c679834b0237 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:00.228581 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c201f-00e3-4c71-a936-c679834b0237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:00.230066 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c201f-00e3-4c71-a936-c679834b0237 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:00.247555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-d97c201f-00e3-4c71-a936-c679834b0237 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:00.247555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:00.247555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:00.247555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:00.247555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:00.247555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:00.247555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:00.247555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:00.247555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:00.247555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:00.247555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:00.247555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:00.247555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:00.247555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:00.247555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:00.248438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:00.248438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:00.248438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:00.248438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:00.248438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:00.248438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:00.248438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:00.248438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:00.248438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:00.248438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:00.248438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:00.248438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:00.248438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:00.248438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:00.248438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:00.248438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:00.248438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:00.249912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:00.249912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:00.249912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:00.249912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:00.249912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 227/485] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:00.254547 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c565521-dafe-4b21-aa30-e0607756258a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:00.400167 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd95fc-9f75-474d-abb3-48d9f88ac20e None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:00.400540 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd95fc-9f75-474d-abb3-48d9f88ac20e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:00.400715 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd95fc-9f75-474d-abb3-48d9f88ac20e None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:00.402109 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 228/486] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:29:00.406637 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c565521-dafe-4b21-aa30-e0607756258a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:00.413692 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04a000-cc05-4582-b10a-452a6991cf40 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:00.413999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04a000-cc05-4582-b10a-452a6991cf40 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:00.414282 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04a000-cc05-4582-b10a-452a6991cf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:00.441528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3e04a000-cc05-4582-b10a-452a6991cf40 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:00.442757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3e04a000-cc05-4582-b10a-452a6991cf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:00.458088 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:00.480207 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1003a1-19a5-44ea-b626-67b577bb9ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:00.480874 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1003a1-19a5-44ea-b626-67b577bb9ceb None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:00.481156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1003a1-19a5-44ea-b626-67b577bb9ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:00.481429 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1003a1-19a5-44ea-b626-67b577bb9ceb None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:00.482237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1003a1-19a5-44ea-b626-67b577bb9ceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:00.482543 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1003a1-19a5-44ea-b626-67b577bb9ceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:00.483993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1003a1-19a5-44ea-b626-67b577bb9ceb None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:00.496933 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 259/487] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:00.498907 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752a631-3eb1-4ad2-8c3e-591cad46a823 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:00.499194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752a631-3eb1-4ad2-8c3e-591cad46a823 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:00.499466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752a631-3eb1-4ad2-8c3e-591cad46a823 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:00.500928 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 260/488] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:00.505862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752a631-3eb1-4ad2-8c3e-591cad46a823 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:00.522035 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 229/489] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:00.531340 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babb58c-b859-4178-9bc2-fd519ab3ef7d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:00.531632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babb58c-b859-4178-9bc2-fd519ab3ef7d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:00.531911 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babb58c-b859-4178-9bc2-fd519ab3ef7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:00.564509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3babb58c-b859-4178-9bc2-fd519ab3ef7d None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:00.566287 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3babb58c-b859-4178-9bc2-fd519ab3ef7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:00.596526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:00.636829 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752a631-3eb1-4ad2-8c3e-591cad46a823 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:00.658447 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 230/490] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:00.659377 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babb58c-b859-4178-9bc2-fd519ab3ef7d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:00.690936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0ab03-8ffc-44f1-b45e-55a7dded8509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:00.691746 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0ab03-8ffc-44f1-b45e-55a7dded8509 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:00.692142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0ab03-8ffc-44f1-b45e-55a7dded8509 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:00.692425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0ab03-8ffc-44f1-b45e-55a7dded8509 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:00.703425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa0ab03-8ffc-44f1-b45e-55a7dded8509 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:00.704903 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa0ab03-8ffc-44f1-b45e-55a7dded8509 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:00.729057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-3fa0ab03-8ffc-44f1-b45e-55a7dded8509 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:29:00.729057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:00.729057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:00.729057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:00.729057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:00.729057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:00.729057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:00.729057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:00.729057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:00.729057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:00.729057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:00.729057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:00.729057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:00.729057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:00.729057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:00.729057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:00.731678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:00.734078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:00.734078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:00.734078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:00.734078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:00.734078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:00.734078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:00.734078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:00.734078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:00.734078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:29:00.734078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:00.734078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 261/491] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jul 12 13:29:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:00.734078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0ab03-8ffc-44f1-b45e-55a7dded8509 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:00.773781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babb58c-b859-4178-9bc2-fd519ab3ef7d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:00.784352 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:00.810460 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda07dc6-0a0e-4cfb-be65-93293c268dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:00.811271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda07dc6-0a0e-4cfb-be65-93293c268dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:00.811553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda07dc6-0a0e-4cfb-be65-93293c268dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:00.811829 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda07dc6-0a0e-4cfb-be65-93293c268dd1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:00.822759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda07dc6-0a0e-4cfb-be65-93293c268dd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:00.831022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-bda07dc6-0a0e-4cfb-be65-93293c268dd1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:00.831022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:00.831022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:00.831022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:00.831022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:00.831022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:00.831022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:00.831022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:00.831022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:00.831022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:00.831022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:00.831022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:00.831022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:00.831022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:00.831022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:00.831022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:00.833741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:00.833741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:00.833741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:00.833741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:00.833741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:00.833741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:00.833741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:00.833741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:00.833741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:00.833741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:00.833741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:00.833741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:00.833741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:00.833741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:00.833741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:00.833741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:00.836162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:00.836162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:00.836162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:00.836162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:00.836162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:00.836162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:00.836162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:00.836162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:00.836162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:00.836162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:00.836162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:00.836162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:00.836162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 231/492] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:00.836162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda07dc6-0a0e-4cfb-be65-93293c268dd1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:00.861373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0ab03-8ffc-44f1-b45e-55a7dded8509 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:00.862666 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:00.899362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7997d97-82ea-4057-bc2a-519ad25d9dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:00.900174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7997d97-82ea-4057-bc2a-519ad25d9dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:00.900460 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7997d97-82ea-4057-bc2a-519ad25d9dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:00.900737 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7997d97-82ea-4057-bc2a-519ad25d9dc0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:00.912034 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7997d97-82ea-4057-bc2a-519ad25d9dc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:00.920561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-f7997d97-82ea-4057-bc2a-519ad25d9dc0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:00.920561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:00.920561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:00.920561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:00.920561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:00.920561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:00.920561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:00.920561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:00.920561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:00.920561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:00.920561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:00.920561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:00.920561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:00.920561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:00.920561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:00.920561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:00.923038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:00.923038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:00.923038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:00.923038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:00.923038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:00.923038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:00.923038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:00.923038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:00.923038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:00.923038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:00.923038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:00.923038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:00.923038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:00.923038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:00.923038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:00.923038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:00.925111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:00.925111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:00.925111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:00.925111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:00.925111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:00.925111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:00.925111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:00.925111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:00.925111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:00.925111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:00.925111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:00.925111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:00.926691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 262/493] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:00.930010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7997d97-82ea-4057-bc2a-519ad25d9dc0 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:00.948712 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda07dc6-0a0e-4cfb-be65-93293c268dd1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:00.993925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f6138-860c-4403-aac7-e08e6ada34c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:00.994628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f6138-860c-4403-aac7-e08e6ada34c3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:00.994910 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f6138-860c-4403-aac7-e08e6ada34c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:00.995185 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f6138-860c-4403-aac7-e08e6ada34c3 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:00.996030 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f6138-860c-4403-aac7-e08e6ada34c3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:00.996341 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f6138-860c-4403-aac7-e08e6ada34c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:00.998579 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f6138-860c-4403-aac7-e08e6ada34c3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:01.040431 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7997d97-82ea-4057-bc2a-519ad25d9dc0 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:01.041985 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:01.054466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 232/494] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jul 12 13:29:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:01.055446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f6138-860c-4403-aac7-e08e6ada34c3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:01.087098 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbe659-7cde-4410-ad8a-76acfae74ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:01.088668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbe659-7cde-4410-ad8a-76acfae74ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:01.089238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbe659-7cde-4410-ad8a-76acfae74ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:01.089769 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbe659-7cde-4410-ad8a-76acfae74ca5 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:01.090705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbe659-7cde-4410-ad8a-76acfae74ca5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:01.091249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbe659-7cde-4410-ad8a-76acfae74ca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:01.094031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbe659-7cde-4410-ad8a-76acfae74ca5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:01.109309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 263/495] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:29:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:01.109309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbe659-7cde-4410-ad8a-76acfae74ca5 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:01.156619 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:01.187538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f6138-860c-4403-aac7-e08e6ada34c3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:01.223728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbe659-7cde-4410-ad8a-76acfae74ca5 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:01.234515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0186e42-03c6-40ea-bfd6-5c5a135fe43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:01.236018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0186e42-03c6-40ea-bfd6-5c5a135fe43a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:01.236508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0186e42-03c6-40ea-bfd6-5c5a135fe43a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:01.237276 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0186e42-03c6-40ea-bfd6-5c5a135fe43a None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:01.238263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0186e42-03c6-40ea-bfd6-5c5a135fe43a None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:01.239030 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0186e42-03c6-40ea-bfd6-5c5a135fe43a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:01.241397 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0186e42-03c6-40ea-bfd6-5c5a135fe43a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:01.251131 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 233/496] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:29:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:01.252459 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0186e42-03c6-40ea-bfd6-5c5a135fe43a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:01.265603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b92dfa-e11b-4d3f-9e09-4f4320ae8a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:01.266299 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b92dfa-e11b-4d3f-9e09-4f4320ae8a6f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:01.266588 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b92dfa-e11b-4d3f-9e09-4f4320ae8a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:01.266864 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b92dfa-e11b-4d3f-9e09-4f4320ae8a6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:01.301775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:01.349719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b92dfa-e11b-4d3f-9e09-4f4320ae8a6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:01.351501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b92dfa-e11b-4d3f-9e09-4f4320ae8a6f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:01.357842 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0186e42-03c6-40ea-bfd6-5c5a135fe43a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:01.391460 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd267d1-e2c0-424b-b56b-7527d100c7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:01.392252 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd267d1-e2c0-424b-b56b-7527d100c7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:01.392537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd267d1-e2c0-424b-b56b-7527d100c7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:01.392814 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd267d1-e2c0-424b-b56b-7527d100c7e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:01.401421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:01.404439 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd267d1-e2c0-424b-b56b-7527d100c7e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:01.406238 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd267d1-e2c0-424b-b56b-7527d100c7e0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:01.424562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-3bd267d1-e2c0-424b-b56b-7527d100c7e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:01.424562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:01.424562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:01.424562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:01.424562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:01.424562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:01.424562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:01.424562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:01.424562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:01.424562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:01.424562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:01.424562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:01.424562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:01.424562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:01.424562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:01.425528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:01.425528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:01.425528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:01.425528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:01.425528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:01.425528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:01.425528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:01.425528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:01.425528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:01.425528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:01.425528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:01.425528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:01.425528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:01.425528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:01.425528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:01.425528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:01.425528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:01.426436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:01.426436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:01.426436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:01.426436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:01.426436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 234/497] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:01.427186 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e575af0-effb-43fc-9dcc-799e7ce623a8 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:01.427294 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e575af0-effb-43fc-9dcc-799e7ce623a8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:01.427539 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e575af0-effb-43fc-9dcc-799e7ce623a8 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:01.428801 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 235/498] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:01.429493 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e575af0-effb-43fc-9dcc-799e7ce623a8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:01.455701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 264/499] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:29:01.457357 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc67c7f-e411-41b8-89c6-c7062c0d2633 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:01.457653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc67c7f-e411-41b8-89c6-c7062c0d2633 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:01.457933 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc67c7f-e411-41b8-89c6-c7062c0d2633 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:01.459350 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 265/500] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:01.460038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc67c7f-e411-41b8-89c6-c7062c0d2633 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:01.525323 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:01.535271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e575af0-effb-43fc-9dcc-799e7ce623a8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:01.565971 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc67c7f-e411-41b8-89c6-c7062c0d2633 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:01.568856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a39b8-9945-46d4-8194-23d12aa4e503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:01.569642 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a39b8-9945-46d4-8194-23d12aa4e503 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:01.569919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a39b8-9945-46d4-8194-23d12aa4e503 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:01.570217 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a39b8-9945-46d4-8194-23d12aa4e503 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:01.581133 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a39b8-9945-46d4-8194-23d12aa4e503 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:01.582610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a39b8-9945-46d4-8194-23d12aa4e503 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:01.600248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-6d2a39b8-9945-46d4-8194-23d12aa4e503 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:01.600248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:01.600248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:01.600248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:01.600248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:01.600248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:01.600248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:01.600248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:01.600248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:01.600248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:01.600248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:01.600248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:01.600248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:01.600248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:01.600248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:01.601206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:01.601206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:01.601206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:01.601206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:01.601206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:01.601206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:01.601206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:01.601206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:01.601206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:01.601206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:01.601206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:01.601206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:01.601206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:01.601206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:01.601206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:01.601206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:01.601206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:01.602073 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:01.602073 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:01.602073 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:01.602073 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:01.602073 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 236/501] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:01.602337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1578f2-d978-4cd1-9fbe-11b853f97077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:01.602422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1578f2-d978-4cd1-9fbe-11b853f97077 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:01.602760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1578f2-d978-4cd1-9fbe-11b853f97077 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:01.602846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4c658-52f8-42b6-b1c6-fcbbaa08e7da None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:01.602960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1578f2-d978-4cd1-9fbe-11b853f97077 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:01.603095 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4c658-52f8-42b6-b1c6-fcbbaa08e7da None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:01.603374 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4c658-52f8-42b6-b1c6-fcbbaa08e7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:01.603683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1578f2-d978-4cd1-9fbe-11b853f97077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:01.603995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1578f2-d978-4cd1-9fbe-11b853f97077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:01.605371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1578f2-d978-4cd1-9fbe-11b853f97077 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:01.617029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 266/502] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:01.617847 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1578f2-d978-4cd1-9fbe-11b853f97077 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:01.620436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-53d4c658-52f8-42b6-b1c6-fcbbaa08e7da None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:01.621797 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-53d4c658-52f8-42b6-b1c6-fcbbaa08e7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:01.662306 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:01.692780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 237/503] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:01.694107 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a508923-7517-4869-9b47-c7c9b24906b3 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:01.694400 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a508923-7517-4869-9b47-c7c9b24906b3 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:01.694683 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a508923-7517-4869-9b47-c7c9b24906b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:01.716901 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4a508923-7517-4869-9b47-c7c9b24906b3 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:01.718052 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4a508923-7517-4869-9b47-c7c9b24906b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:01.732807 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1578f2-d978-4cd1-9fbe-11b853f97077 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:01.756795 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:01.768846 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8a7dd-ccb3-4716-93c3-6ee196ffdbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:01.769657 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8a7dd-ccb3-4716-93c3-6ee196ffdbba None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:01.769947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8a7dd-ccb3-4716-93c3-6ee196ffdbba None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:01.770253 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8a7dd-ccb3-4716-93c3-6ee196ffdbba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:01.781774 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a8a7dd-ccb3-4716-93c3-6ee196ffdbba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:01.783285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a8a7dd-ccb3-4716-93c3-6ee196ffdbba None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:01.785686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 238/504] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:01.786561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a508923-7517-4869-9b47-c7c9b24906b3 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:01.800957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-39a8a7dd-ccb3-4716-93c3-6ee196ffdbba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:01.800957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:01.800957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:01.800957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:01.800957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:01.800957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:01.800957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:01.800957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:01.800957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:01.800957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:01.800957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:01.800957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:01.800957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:01.800957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:01.800957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:01.801873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:01.801873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:01.801873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:01.801873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:01.801873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:01.801873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:01.801873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:01.801873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:01.801873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:01.801873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:01.801873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:01.801873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:01.801873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:01.801873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:01.801873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:01.801873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:01.801873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:01.802750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:01.802750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:01.802750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:01.802750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:01.808003 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 267/505] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:01.808003 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8a7dd-ccb3-4716-93c3-6ee196ffdbba None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:01.885984 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a508923-7517-4869-9b47-c7c9b24906b3 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:01.888889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:01.907623 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8a7dd-ccb3-4716-93c3-6ee196ffdbba None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:01.921001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28620262-51a0-4531-9e43-bfa36e1a1cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:01.921657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28620262-51a0-4531-9e43-bfa36e1a1cfc None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:01.921935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28620262-51a0-4531-9e43-bfa36e1a1cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:01.922237 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28620262-51a0-4531-9e43-bfa36e1a1cfc None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:01.923040 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28620262-51a0-4531-9e43-bfa36e1a1cfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:01.923246 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28620262-51a0-4531-9e43-bfa36e1a1cfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:01.924607 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28620262-51a0-4531-9e43-bfa36e1a1cfc None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:01.936638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 239/506] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:01.937401 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28620262-51a0-4531-9e43-bfa36e1a1cfc None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:01.942664 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b2b43-27de-4900-9b1a-4a2a3112ea5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:01.943466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b2b43-27de-4900-9b1a-4a2a3112ea5c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:01.943748 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b2b43-27de-4900-9b1a-4a2a3112ea5c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:01.944022 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b2b43-27de-4900-9b1a-4a2a3112ea5c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:01.952932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b2b43-27de-4900-9b1a-4a2a3112ea5c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:01.954435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b2b43-27de-4900-9b1a-4a2a3112ea5c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:01.968391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-ca1b2b43-27de-4900-9b1a-4a2a3112ea5c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 13:29:01.968391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:01.968391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:01.968391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:01.968391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:01.968391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:01.968391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:01.968391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:01.968391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:01.968391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:01.968391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:01.968391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:01.968391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:01.968391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:01.968391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:01.968391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:01.969406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:01.970362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:01.970362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:01.970362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:01.970362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:01.970362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:01.970362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:01.970362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:01.970362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:01.970362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 13:29:01.970362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:01.970362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 268/507] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:29:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:01.970362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b2b43-27de-4900-9b1a-4a2a3112ea5c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:02.005082 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:02.042331 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28620262-51a0-4531-9e43-bfa36e1a1cfc None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:02.070109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b2b43-27de-4900-9b1a-4a2a3112ea5c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:02.075903 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3042cde-1868-4136-a27f-b5c88dde96a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:02.076746 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3042cde-1868-4136-a27f-b5c88dde96a6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:02.077030 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3042cde-1868-4136-a27f-b5c88dde96a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:02.077311 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3042cde-1868-4136-a27f-b5c88dde96a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:02.087644 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3042cde-1868-4136-a27f-b5c88dde96a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:02.095452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-a3042cde-1868-4136-a27f-b5c88dde96a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:02.095452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:02.095452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:02.095452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:02.095452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:02.095452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:02.095452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:02.095452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:02.095452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:02.095452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:02.095452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:02.095452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:02.095452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:02.095452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:02.095452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:02.095452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:02.096428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:02.096428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:02.096428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:02.096428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:02.096428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:02.096428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:02.096428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:02.096428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:02.096428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:02.096428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:02.096428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:02.096428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:02.096428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:02.096428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:02.096428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:02.096428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:02.097225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:02.097225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:02.097225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:02.097225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:02.097225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:02.097225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:02.097225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:02.097225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:02.097225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:02.097225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:02.097225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:02.097225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:02.097225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 240/508] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:02.098417 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3042cde-1868-4136-a27f-b5c88dde96a6 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:02.107067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56345400-e960-4383-aa28-807ef91fe4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:02.107730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56345400-e960-4383-aa28-807ef91fe4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:02.108012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56345400-e960-4383-aa28-807ef91fe4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:02.108285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56345400-e960-4383-aa28-807ef91fe4c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:02.109003 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56345400-e960-4383-aa28-807ef91fe4c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:02.109307 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56345400-e960-4383-aa28-807ef91fe4c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:02.110713 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56345400-e960-4383-aa28-807ef91fe4c0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:02.122738 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 269/509] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:02.123521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56345400-e960-4383-aa28-807ef91fe4c0 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:02.123874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:02.198176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3042cde-1868-4136-a27f-b5c88dde96a6 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:02.215739 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:02.221998 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56345400-e960-4383-aa28-807ef91fe4c0 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:02.231789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3d6ec-8418-4b6f-ad61-a7bf4bcbb76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:02.232583 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3d6ec-8418-4b6f-ad61-a7bf4bcbb76d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:02.232871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3d6ec-8418-4b6f-ad61-a7bf4bcbb76d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:02.233151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3d6ec-8418-4b6f-ad61-a7bf4bcbb76d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:02.243852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3d6ec-8418-4b6f-ad61-a7bf4bcbb76d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:02.245308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3d6ec-8418-4b6f-ad61-a7bf4bcbb76d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:02.255708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42ee31-6dab-4844-ae2b-7973231f5ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:02.256525 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42ee31-6dab-4844-ae2b-7973231f5ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:02.256795 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42ee31-6dab-4844-ae2b-7973231f5ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:02.257072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42ee31-6dab-4844-ae2b-7973231f5ed4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:02.263384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-5ed3d6ec-8418-4b6f-ad61-a7bf4bcbb76d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:02.263384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:02.263384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:02.263384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:02.263384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:02.263384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:02.263384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:02.263384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:02.263384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:02.263384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:02.263384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:02.263384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:02.263384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:02.263384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:02.263384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:02.264243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:02.264243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:02.264243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:02.264243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:02.264243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:02.264243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:02.264243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:02.264243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:02.264243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:02.264243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:02.264243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:02.264243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:02.264243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:02.264243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:02.264243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:02.264243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:02.264243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:02.265080 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:02.265080 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:02.265080 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:02.265080 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:02.265080 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 241/510] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:02.265363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3d6ec-8418-4b6f-ad61-a7bf4bcbb76d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:02.266803 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42ee31-6dab-4844-ae2b-7973231f5ed4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:02.268419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42ee31-6dab-4844-ae2b-7973231f5ed4 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:02.283440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-2c42ee31-6dab-4844-ae2b-7973231f5ed4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 13:29:02.283440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:02.283440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:02.283440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:02.283440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:02.283440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:02.283440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:02.283440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:02.283440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:02.283440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:02.283440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:02.283440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:02.283440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:02.283440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:02.283440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:02.283440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:02.284554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:02.285630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:02.285630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:02.285630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:02.285630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:02.285630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:02.285630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:02.285630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:02.285630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:02.285630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 13:29:02.285630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:02.285630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 270/511] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:29:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:02.285630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42ee31-6dab-4844-ae2b-7973231f5ed4 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:02.343924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:02.367701 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3d6ec-8418-4b6f-ad61-a7bf4bcbb76d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:02.386835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42ee31-6dab-4844-ae2b-7973231f5ed4 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:02.402188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7aaa9-967a-4fc6-bd47-8abe5213db66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:02.403036 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7aaa9-967a-4fc6-bd47-8abe5213db66 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:02.403457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7aaa9-967a-4fc6-bd47-8abe5213db66 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:02.403828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7aaa9-967a-4fc6-bd47-8abe5213db66 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:02.404757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7aaa9-967a-4fc6-bd47-8abe5213db66 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:02.405193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7aaa9-967a-4fc6-bd47-8abe5213db66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:02.406760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7aaa9-967a-4fc6-bd47-8abe5213db66 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:02.422930 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02638afe-939c-43e7-87cd-1d10e4338d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:02.423595 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02638afe-939c-43e7-87cd-1d10e4338d71 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:02.423879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02638afe-939c-43e7-87cd-1d10e4338d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:02.424152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02638afe-939c-43e7-87cd-1d10e4338d71 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:02.424417 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 242/512] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Jul 12 13:29:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:02.424903 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02638afe-939c-43e7-87cd-1d10e4338d71 None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:02.425209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7aaa9-967a-4fc6-bd47-8abe5213db66 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:02.425321 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02638afe-939c-43e7-87cd-1d10e4338d71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:02.426630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02638afe-939c-43e7-87cd-1d10e4338d71 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:02.435251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 271/513] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:29:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:02.436043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02638afe-939c-43e7-87cd-1d10e4338d71 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:02.454176 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:02.528689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7aaa9-967a-4fc6-bd47-8abe5213db66 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:02.535782 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02638afe-939c-43e7-87cd-1d10e4338d71 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:02.564546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87184144-03f3-4500-a8ac-78d80c462557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:02.565344 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87184144-03f3-4500-a8ac-78d80c462557 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:02.565598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87184144-03f3-4500-a8ac-78d80c462557 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:02.565867 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87184144-03f3-4500-a8ac-78d80c462557 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:02.567858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:02.572411 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd75e60-d36b-4aa8-a195-c07a440b24bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:02.572943 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87184144-03f3-4500-a8ac-78d80c462557 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:02.573129 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd75e60-d36b-4aa8-a195-c07a440b24bc None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:02.573514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd75e60-d36b-4aa8-a195-c07a440b24bc None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:02.573888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd75e60-d36b-4aa8-a195-c07a440b24bc None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:02.574640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87184144-03f3-4500-a8ac-78d80c462557 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:02.574801 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd75e60-d36b-4aa8-a195-c07a440b24bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:02.575186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd75e60-d36b-4aa8-a195-c07a440b24bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:02.576841 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd75e60-d36b-4aa8-a195-c07a440b24bc None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:02.588938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 272/514] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:02.589102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-87184144-03f3-4500-a8ac-78d80c462557 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:02.589102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:02.589102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:02.589102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:02.589102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:02.589102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:02.589102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:02.589102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:02.589102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:02.589102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:02.589102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:02.589102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:02.589102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:02.589102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:02.589102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:02.589882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:02.589882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:02.589882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:02.589882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:02.589882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:02.589882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:02.589882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:02.589882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:02.589882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:02.589882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:02.589882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:02.589882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:02.589882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:02.589882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:02.589882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:02.589882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:02.589882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:02.590789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:02.590789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:02.590789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:02.590789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:02.590789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:02.590789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:02.590789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:02.590789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:02.592129 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd75e60-d36b-4aa8-a195-c07a440b24bc None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:02.592298 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 243/515] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:02.592298 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87184144-03f3-4500-a8ac-78d80c462557 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:02.674402 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:02.695436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87184144-03f3-4500-a8ac-78d80c462557 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:02.698042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd75e60-d36b-4aa8-a195-c07a440b24bc None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:02.730167 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e1e68-3197-4f5b-9417-3eb3437e1826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:02.730818 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e1e68-3197-4f5b-9417-3eb3437e1826 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:02.731103 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e1e68-3197-4f5b-9417-3eb3437e1826 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:02.731381 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e1e68-3197-4f5b-9417-3eb3437e1826 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:02.732857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47b832-bbd0-4334-b2d5-1e60575217d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:02.733525 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47b832-bbd0-4334-b2d5-1e60575217d7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:02.733813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47b832-bbd0-4334-b2d5-1e60575217d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:02.734123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47b832-bbd0-4334-b2d5-1e60575217d7 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:02.734957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47b832-bbd0-4334-b2d5-1e60575217d7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:02.735274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47b832-bbd0-4334-b2d5-1e60575217d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:02.736725 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47b832-bbd0-4334-b2d5-1e60575217d7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:02.772457 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 273/516] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Jul 12 13:29:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:02.773173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47b832-bbd0-4334-b2d5-1e60575217d7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:02.794749 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:02.803486 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e1e68-3197-4f5b-9417-3eb3437e1826 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:02.804952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e1e68-3197-4f5b-9417-3eb3437e1826 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:02.876209 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47b832-bbd0-4334-b2d5-1e60575217d7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:02.903039 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 244/517] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:29:02.904401 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e1e68-3197-4f5b-9417-3eb3437e1826 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:02.912031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cff39-b810-47c3-be4e-495b202801a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:02.912824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cff39-b810-47c3-be4e-495b202801a3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:02.913240 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cff39-b810-47c3-be4e-495b202801a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:02.913379 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cff39-b810-47c3-be4e-495b202801a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:02.916206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:02.925097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cff39-b810-47c3-be4e-495b202801a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:02.926623 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cff39-b810-47c3-be4e-495b202801a3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:02.944309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-9f9cff39-b810-47c3-be4e-495b202801a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:02.944309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:02.944309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:02.944309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:02.944309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:02.944309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:02.944309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:02.944309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:02.944309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:02.944309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:02.944309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:02.944309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:02.944309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:02.944309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:02.944309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:02.945348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:02.945348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:02.945348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:02.945348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:02.945348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:02.945348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:02.945348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:02.945348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:02.945348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:02.945348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:02.945348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:02.945348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:02.945348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:02.945348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:02.945348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:02.945348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:02.945348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:02.946265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:02.946265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:02.946265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:02.946265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:02.946265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 274/518] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:02.946265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cff39-b810-47c3-be4e-495b202801a3 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:02.999996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:03.007304 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e1e68-3197-4f5b-9417-3eb3437e1826 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:03.041563 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bfb08-e561-4125-8cca-2b2c02765087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:03.042271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bfb08-e561-4125-8cca-2b2c02765087 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:03.042557 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bfb08-e561-4125-8cca-2b2c02765087 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:03.042833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bfb08-e561-4125-8cca-2b2c02765087 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:03.043538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bfb08-e561-4125-8cca-2b2c02765087 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:03.045059 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bfb08-e561-4125-8cca-2b2c02765087 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:03.046868 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cff39-b810-47c3-be4e-495b202801a3 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:03.064053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 245/519] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:29:02 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:03.064959 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bfb08-e561-4125-8cca-2b2c02765087 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:03.083032 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65d48c-459c-443c-857f-bf5cc8aea812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:03.083726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65d48c-459c-443c-857f-bf5cc8aea812 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:03.084004 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65d48c-459c-443c-857f-bf5cc8aea812 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:03.084348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65d48c-459c-443c-857f-bf5cc8aea812 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:03.085158 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65d48c-459c-443c-857f-bf5cc8aea812 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:03.085470 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65d48c-459c-443c-857f-bf5cc8aea812 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:03.087025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65d48c-459c-443c-857f-bf5cc8aea812 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:03.114225 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 275/520] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:03.114225 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65d48c-459c-443c-857f-bf5cc8aea812 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:03.166376 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:03.192005 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bfb08-e561-4125-8cca-2b2c02765087 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:03.224881 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65d48c-459c-443c-857f-bf5cc8aea812 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:03.232006 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17769784-5bbd-43ce-99d9-6b455d54c103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:03.232658 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17769784-5bbd-43ce-99d9-6b455d54c103 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:03.232941 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17769784-5bbd-43ce-99d9-6b455d54c103 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:03.233211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17769784-5bbd-43ce-99d9-6b455d54c103 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:03.253112 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:03.272750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f79806-5b93-474b-a749-3db7db4d36b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:03.273584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f79806-5b93-474b-a749-3db7db4d36b9 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:03.273852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f79806-5b93-474b-a749-3db7db4d36b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:03.274177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f79806-5b93-474b-a749-3db7db4d36b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:03.289695 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f79806-5b93-474b-a749-3db7db4d36b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:03.291222 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f79806-5b93-474b-a749-3db7db4d36b9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:03.309688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-14f79806-5b93-474b-a749-3db7db4d36b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:03.309688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:03.309688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:03.309688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:03.309688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:03.309688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:03.309688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:03.309688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:03.309688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:03.309688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:03.309688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:03.309688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:03.309688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:03.309688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:03.309688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:03.310620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:03.310620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:03.310620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:03.310620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:03.310620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:03.310620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:03.310620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:03.310620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:03.310620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:03.310620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:03.310620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:03.310620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:03.310620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:03.310620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:03.310620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:03.310620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:03.310620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:03.311474 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:03.311474 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:03.311474 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:03.311474 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:03.311694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17769784-5bbd-43ce-99d9-6b455d54c103 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:03.311764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 276/521] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:03.311827 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17769784-5bbd-43ce-99d9-6b455d54c103 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:03.312372 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f79806-5b93-474b-a749-3db7db4d36b9 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:03.420733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:03.424911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f79806-5b93-474b-a749-3db7db4d36b9 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:03.427992 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 246/522] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:29:03.433132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17769784-5bbd-43ce-99d9-6b455d54c103 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:03.465485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d35765-1c95-443e-b768-bb769f0867e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:03.466186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d35765-1c95-443e-b768-bb769f0867e9 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:03.466472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d35765-1c95-443e-b768-bb769f0867e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:03.466748 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d35765-1c95-443e-b768-bb769f0867e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:03.467474 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d35765-1c95-443e-b768-bb769f0867e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:03.467782 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d35765-1c95-443e-b768-bb769f0867e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:03.469213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d35765-1c95-443e-b768-bb769f0867e9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:03.483871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 277/523] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:03.488906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d35765-1c95-443e-b768-bb769f0867e9 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:03.492353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:03.541571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17769784-5bbd-43ce-99d9-6b455d54c103 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:03.582002 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf4b5c-680a-4b0a-8e68-c065f4ab7c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:03.582640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf4b5c-680a-4b0a-8e68-c065f4ab7c20 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:03.582842 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf4b5c-680a-4b0a-8e68-c065f4ab7c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:03.583114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf4b5c-680a-4b0a-8e68-c065f4ab7c20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:03.583932 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf4b5c-680a-4b0a-8e68-c065f4ab7c20 None admin] RBAC: Adding query filter params (role.id=057d9335b3b1412ebfb6bfb27f02f96a, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=0d3915cde0d04bcfb9b5ce8c89af1e5b) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:03.584235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf4b5c-680a-4b0a-8e68-c065f4ab7c20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:03.585669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf4b5c-680a-4b0a-8e68-c065f4ab7c20 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:03.601988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 247/524] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:03 2022] GET /identity/v3/role_assignments?user.id=0d3915cde0d04bcfb9b5ce8c89af1e5b&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=057d9335b3b1412ebfb6bfb27f02f96a => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:03.601988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf4b5c-680a-4b0a-8e68-c065f4ab7c20 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:03.605024 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d35765-1c95-443e-b768-bb769f0867e9 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:03.660503 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24a33c-370f-4475-83ee-6052ad7a95a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:03.662158 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24a33c-370f-4475-83ee-6052ad7a95a9 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:03.662529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24a33c-370f-4475-83ee-6052ad7a95a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:03.662929 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24a33c-370f-4475-83ee-6052ad7a95a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:03.664295 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24a33c-370f-4475-83ee-6052ad7a95a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:03.664686 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24a33c-370f-4475-83ee-6052ad7a95a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:03.667064 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24a33c-370f-4475-83ee-6052ad7a95a9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:03.685846 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 278/525] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:03.687286 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24a33c-370f-4475-83ee-6052ad7a95a9 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:03.719976 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:03.735499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf4b5c-680a-4b0a-8e68-c065f4ab7c20 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:03.759694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:03.784964 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acb1d7-16d3-4c0d-bc9f-bb2d768a477b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:03.785780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acb1d7-16d3-4c0d-bc9f-bb2d768a477b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:03.786106 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acb1d7-16d3-4c0d-bc9f-bb2d768a477b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:03.786378 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acb1d7-16d3-4c0d-bc9f-bb2d768a477b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:03.798607 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0acb1d7-16d3-4c0d-bc9f-bb2d768a477b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:03.800068 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0acb1d7-16d3-4c0d-bc9f-bb2d768a477b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:03.819891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-a0acb1d7-16d3-4c0d-bc9f-bb2d768a477b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:03.819891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:03.819891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:03.819891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:03.819891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:03.819891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:03.819891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:03.819891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:03.819891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:03.819891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:03.819891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:03.819891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:03.819891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:03.819891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:03.819891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:03.820821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:03.820821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:03.820821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:03.820821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:03.820821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:03.820821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:03.820821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:03.820821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:03.820821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:03.820821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:03.820821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:03.820821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:03.820821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:03.820821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:03.820821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:03.820821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:03.820821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:03.821668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:03.821668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:03.821668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:03.821668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:03.821900 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24a33c-370f-4475-83ee-6052ad7a95a9 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:03.825694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 248/526] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:03.826736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acb1d7-16d3-4c0d-bc9f-bb2d768a477b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:03.865902 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2bdaa-be97-4489-8442-7517444a0aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:03.867235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2bdaa-be97-4489-8442-7517444a0aad None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:03.867670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2bdaa-be97-4489-8442-7517444a0aad None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:03.868051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2bdaa-be97-4489-8442-7517444a0aad None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:03.879727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db2bdaa-be97-4489-8442-7517444a0aad None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:03.882000 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db2bdaa-be97-4489-8442-7517444a0aad None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:03.903010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-8db2bdaa-be97-4489-8442-7517444a0aad None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 13:29:03.903010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:03.903010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:03.903010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:03.903010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:03.903010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:03.903010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:03.903010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:03.903010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:03.903010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:03.903010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:03.903010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:03.903010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:03.903010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:03.903010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:03.903010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:03.904238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:03.905132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:03.905132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:03.905132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:03.905132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:03.905132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:03.905132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:03.905132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:03.905132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:03.905132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 13:29:03.905132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:03.905132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 279/527] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:03.910124 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2bdaa-be97-4489-8442-7517444a0aad None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:03.960833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acb1d7-16d3-4c0d-bc9f-bb2d768a477b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:03.998981 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df003d-839f-4fb1-bc71-63b5ebf50677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:04.000068 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df003d-839f-4fb1-bc71-63b5ebf50677 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:04.000546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df003d-839f-4fb1-bc71-63b5ebf50677 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:04.000986 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df003d-839f-4fb1-bc71-63b5ebf50677 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:04.008813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df003d-839f-4fb1-bc71-63b5ebf50677 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:04.010559 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df003d-839f-4fb1-bc71-63b5ebf50677 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:04.021026 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:04.025891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-22df003d-839f-4fb1-bc71-63b5ebf50677 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:04.025891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:04.025891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:04.025891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:04.025891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:04.025891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:04.025891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:04.025891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:04.025891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:04.025891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:04.025891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:04.025891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:04.025891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:04.025891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:04.025891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:04.026831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:04.026831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:04.026831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:04.026831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:04.026831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:04.026831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:04.026831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:04.026831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:04.026831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:04.026831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:04.026831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:04.026831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:04.026831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:04.026831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:04.026831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:04.026831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:04.026831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:04.028165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:04.028165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:04.028165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:04.028165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:04.028165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:04.028165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:04.028165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:04.028165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:04.030499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 249/528] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:04.031517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df003d-839f-4fb1-bc71-63b5ebf50677 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:04.055962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2bdaa-be97-4489-8442-7517444a0aad None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:04.069339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:04.101831 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5250591-73a4-4ff2-9efc-5f8ac333c40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:04.102542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5250591-73a4-4ff2-9efc-5f8ac333c40a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:04.103230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5250591-73a4-4ff2-9efc-5f8ac333c40a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:04.103523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5250591-73a4-4ff2-9efc-5f8ac333c40a None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:04.104263 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5250591-73a4-4ff2-9efc-5f8ac333c40a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:04.104579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5250591-73a4-4ff2-9efc-5f8ac333c40a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:04.106014 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5250591-73a4-4ff2-9efc-5f8ac333c40a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:04.123531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 280/529] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:04.124470 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5250591-73a4-4ff2-9efc-5f8ac333c40a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:04.171153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df003d-839f-4fb1-bc71-63b5ebf50677 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:04.213739 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e858ec-f9b2-47c0-a1dc-aba125e4be46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:04.214476 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e858ec-f9b2-47c0-a1dc-aba125e4be46 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:04.214761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e858ec-f9b2-47c0-a1dc-aba125e4be46 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:04.215038 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e858ec-f9b2-47c0-a1dc-aba125e4be46 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:04.216387 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e858ec-f9b2-47c0-a1dc-aba125e4be46 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:04.216698 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e858ec-f9b2-47c0-a1dc-aba125e4be46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:04.218188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e858ec-f9b2-47c0-a1dc-aba125e4be46 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:04.237114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 250/530] 10.222.0.52 () {56 vars in 1139 bytes} [Tue Jul 12 13:29:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:04.238082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e858ec-f9b2-47c0-a1dc-aba125e4be46 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:04.238650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5250591-73a4-4ff2-9efc-5f8ac333c40a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:04.275484 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f924ff9-a5b2-460a-a4bf-cecbbd369c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:04.276143 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f924ff9-a5b2-460a-a4bf-cecbbd369c41 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:04.276423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f924ff9-a5b2-460a-a4bf-cecbbd369c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:04.276696 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f924ff9-a5b2-460a-a4bf-cecbbd369c41 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:04.277488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f924ff9-a5b2-460a-a4bf-cecbbd369c41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:04.277796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f924ff9-a5b2-460a-a4bf-cecbbd369c41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:04.279814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f924ff9-a5b2-460a-a4bf-cecbbd369c41 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:04.289469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:04.294523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 281/531] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:04.296250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f924ff9-a5b2-460a-a4bf-cecbbd369c41 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:04.333052 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:04.346154 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e858ec-f9b2-47c0-a1dc-aba125e4be46 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:04.384905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5088dff-035c-4a43-9e66-ad76385d833b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:04.385637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5088dff-035c-4a43-9e66-ad76385d833b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:04.385920 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5088dff-035c-4a43-9e66-ad76385d833b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:04.386225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5088dff-035c-4a43-9e66-ad76385d833b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:04.393852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5088dff-035c-4a43-9e66-ad76385d833b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:04.395482 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5088dff-035c-4a43-9e66-ad76385d833b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:04.400237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f924ff9-a5b2-460a-a4bf-cecbbd369c41 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:04.411345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-f5088dff-035c-4a43-9e66-ad76385d833b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:04.411345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:04.411345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:04.411345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:04.411345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:04.411345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:04.411345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:04.411345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:04.411345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:04.411345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:04.411345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:04.411345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:04.411345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:04.411345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:04.411345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:04.412413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:04.412413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:04.412413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:04.412413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:04.412413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:04.412413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:04.412413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:04.412413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:04.412413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:04.412413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:04.412413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:04.412413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:04.412413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:04.412413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:04.412413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:04.412413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:04.412413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:04.413299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:04.413299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:04.413299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:04.413299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:04.413299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:04.413299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:04.413299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:04.413299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:04.413299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 251/532] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:04.413760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5088dff-035c-4a43-9e66-ad76385d833b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:04.444246 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906d750-52ad-41e3-a9f4-24928465f330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:04.445273 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906d750-52ad-41e3-a9f4-24928465f330 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:04.445796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906d750-52ad-41e3-a9f4-24928465f330 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:04.446306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906d750-52ad-41e3-a9f4-24928465f330 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:04.515577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:04.518618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906d750-52ad-41e3-a9f4-24928465f330 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:04.520363 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906d750-52ad-41e3-a9f4-24928465f330 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:04.524749 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5088dff-035c-4a43-9e66-ad76385d833b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:04.560950 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53609241-dbc4-410f-86fc-bcf966a399d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:04.561843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53609241-dbc4-410f-86fc-bcf966a399d1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:04.562417 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53609241-dbc4-410f-86fc-bcf966a399d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:04.562948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53609241-dbc4-410f-86fc-bcf966a399d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:04.563921 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53609241-dbc4-410f-86fc-bcf966a399d1 None admin] RBAC: Adding query filter params (role.id=057d9335b3b1412ebfb6bfb27f02f96a, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=ebb4da7b00c64fafbb3724bae25b2853) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:04.564494 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53609241-dbc4-410f-86fc-bcf966a399d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:04.566106 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53609241-dbc4-410f-86fc-bcf966a399d1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:04.572644 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 252/533] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:04 2022] GET /identity/v3/role_assignments?user.id=ebb4da7b00c64fafbb3724bae25b2853&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=057d9335b3b1412ebfb6bfb27f02f96a => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:04.573710 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53609241-dbc4-410f-86fc-bcf966a399d1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:04.618000 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:04.634770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 282/534] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:29:04.638155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906d750-52ad-41e3-a9f4-24928465f330 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:04.790134 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53609241-dbc4-410f-86fc-bcf966a399d1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:04.809527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:04.814284 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906d750-52ad-41e3-a9f4-24928465f330 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:04.861124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a322ad0-5377-4339-b347-e50ef11f53fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:04.861793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a322ad0-5377-4339-b347-e50ef11f53fd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:04.862102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a322ad0-5377-4339-b347-e50ef11f53fd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:04.862376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a322ad0-5377-4339-b347-e50ef11f53fd None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:04.863164 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a322ad0-5377-4339-b347-e50ef11f53fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:04.863468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a322ad0-5377-4339-b347-e50ef11f53fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:04.864916 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a322ad0-5377-4339-b347-e50ef11f53fd None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:04.883973 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b384f1-4d44-4cec-9b8b-47306ea48f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:04.884784 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b384f1-4d44-4cec-9b8b-47306ea48f0b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:04.885066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b384f1-4d44-4cec-9b8b-47306ea48f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:04.885342 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b384f1-4d44-4cec-9b8b-47306ea48f0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:04.900809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 253/535] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:04.905046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b384f1-4d44-4cec-9b8b-47306ea48f0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:04.906575 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b384f1-4d44-4cec-9b8b-47306ea48f0b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:04.911067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a322ad0-5377-4339-b347-e50ef11f53fd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:04.927119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-57b384f1-4d44-4cec-9b8b-47306ea48f0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:04.927119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:04.927119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:04.927119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:04.927119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:04.927119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:04.927119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:04.927119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:04.927119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:04.927119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:04.927119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:04.927119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:04.927119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:04.927119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:04.927119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:04.928028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:04.928028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:04.928028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:04.928028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:04.928028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:04.928028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:04.928028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:04.928028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:04.928028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:04.928028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:04.928028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:04.928028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:04.928028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:04.928028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:04.928028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:04.928028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:04.928028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:04.928877 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:04.928877 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:04.928877 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:04.928877 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:04.928877 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 283/536] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:04.951191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b384f1-4d44-4cec-9b8b-47306ea48f0b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:04.993159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:05.042893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a322ad0-5377-4339-b347-e50ef11f53fd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:05.081256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60876914-91ff-4bd0-8fba-4362d49ee0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:05.082304 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60876914-91ff-4bd0-8fba-4362d49ee0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:05.082730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60876914-91ff-4bd0-8fba-4362d49ee0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:05.083138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60876914-91ff-4bd0-8fba-4362d49ee0c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:05.084443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60876914-91ff-4bd0-8fba-4362d49ee0c7 None admin] RBAC: Adding query filter params (role.id=057d9335b3b1412ebfb6bfb27f02f96a, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=35336384bbca475082be349df406ae3f) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:05.084919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60876914-91ff-4bd0-8fba-4362d49ee0c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:05.087285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60876914-91ff-4bd0-8fba-4362d49ee0c7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:05.097229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 254/537] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:04 2022] GET /identity/v3/role_assignments?user.id=35336384bbca475082be349df406ae3f&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=057d9335b3b1412ebfb6bfb27f02f96a => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:05.174844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b384f1-4d44-4cec-9b8b-47306ea48f0b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:05.233769 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70251f91-98ba-4d2c-a54b-bf62476043b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:05.234492 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70251f91-98ba-4d2c-a54b-bf62476043b7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:05.234773 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70251f91-98ba-4d2c-a54b-bf62476043b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:05.235047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70251f91-98ba-4d2c-a54b-bf62476043b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:05.235791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70251f91-98ba-4d2c-a54b-bf62476043b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:05.236100 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70251f91-98ba-4d2c-a54b-bf62476043b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:05.237549 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70251f91-98ba-4d2c-a54b-bf62476043b7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:05.256330 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 284/538] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:05.261473 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60876914-91ff-4bd0-8fba-4362d49ee0c7 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:05.298021 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4113e-3eb1-4a51-a39a-3f306396b913 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:05.298707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4113e-3eb1-4a51-a39a-3f306396b913 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:05.299215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4113e-3eb1-4a51-a39a-3f306396b913 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:05.301118 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 285/539] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:05.306916 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53356c-5dc8-4fc8-8557-245347049034 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:05.307602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53356c-5dc8-4fc8-8557-245347049034 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:05.308088 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53356c-5dc8-4fc8-8557-245347049034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:05.328073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-2c53356c-5dc8-4fc8-8557-245347049034 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:05.330230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2c53356c-5dc8-4fc8-8557-245347049034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:05.341476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:05.426896 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60876914-91ff-4bd0-8fba-4362d49ee0c7 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:05.428747 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 286/540] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:05.476637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:05.486464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71070cc6-8303-44b2-9f57-613d439f0fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:05.487615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71070cc6-8303-44b2-9f57-613d439f0fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:05.488138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71070cc6-8303-44b2-9f57-613d439f0fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:05.488791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71070cc6-8303-44b2-9f57-613d439f0fa0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:05.496840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71070cc6-8303-44b2-9f57-613d439f0fa0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:05.502499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71070cc6-8303-44b2-9f57-613d439f0fa0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:05.504246 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53356c-5dc8-4fc8-8557-245347049034 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:05.524026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-71070cc6-8303-44b2-9f57-613d439f0fa0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:05.524026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:05.524026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:05.524026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:05.524026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:05.524026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:05.524026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:05.524026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:05.524026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:05.524026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:05.524026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:05.524026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:05.524026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:05.524026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:05.524026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:05.524919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:05.524919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:05.524919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:05.524919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:05.524919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:05.524919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:05.524919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:05.524919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:05.524919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:05.524919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:05.524919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:05.524919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:05.524919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:05.524919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:05.524919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:05.524919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:05.524919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:05.525759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:05.525759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:05.525759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:05.525759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:05.525759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:05.525759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:05.525759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:05.525759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:05.534257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 255/541] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:05.539157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71070cc6-8303-44b2-9f57-613d439f0fa0 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:05.666223 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71070cc6-8303-44b2-9f57-613d439f0fa0 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:05.706012 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b31e12-3d2c-45f4-90b0-bba09369aab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:05.706999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b31e12-3d2c-45f4-90b0-bba09369aab5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:05.707510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b31e12-3d2c-45f4-90b0-bba09369aab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:05.707987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b31e12-3d2c-45f4-90b0-bba09369aab5 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:05.708971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b31e12-3d2c-45f4-90b0-bba09369aab5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:05.709481 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b31e12-3d2c-45f4-90b0-bba09369aab5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:05.711253 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b31e12-3d2c-45f4-90b0-bba09369aab5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:05.725195 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 256/542] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:05.733371 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b31e12-3d2c-45f4-90b0-bba09369aab5 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:05.741191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53356c-5dc8-4fc8-8557-245347049034 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:05.779651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d15ef-1e48-457d-aab2-82c9aeca3713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:05.780464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d15ef-1e48-457d-aab2-82c9aeca3713 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:05.780750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d15ef-1e48-457d-aab2-82c9aeca3713 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:05.781028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d15ef-1e48-457d-aab2-82c9aeca3713 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:05.792983 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d15ef-1e48-457d-aab2-82c9aeca3713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:05.802661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-fd7d15ef-1e48-457d-aab2-82c9aeca3713 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:05.802661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:05.802661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:05.802661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:05.802661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:05.802661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:05.802661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:05.802661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:05.802661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:05.802661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:05.802661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:05.802661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:05.802661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:05.802661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:05.802661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:05.802661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:05.803626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:05.803626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:05.803626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:05.803626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:05.803626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:05.803626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:05.803626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:05.803626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:05.803626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:05.803626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:05.803626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:05.803626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:05.803626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:05.803626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:05.803626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:05.803626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:05.804419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:05.804419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:05.804419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:05.804419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:05.804419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:05.804419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:05.804419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:05.804419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:05.804419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:05.804419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:05.804419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:05.804419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:05.804419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 287/543] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:05.810893 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d15ef-1e48-457d-aab2-82c9aeca3713 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:05.826300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:05.859636 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:05.912259 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b31e12-3d2c-45f4-90b0-bba09369aab5 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:05.947504 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d15ef-1e48-457d-aab2-82c9aeca3713 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:05.948889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26aa3c-7319-4b0d-bd8d-fe20648264a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:05.949861 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26aa3c-7319-4b0d-bd8d-fe20648264a5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:05.950209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26aa3c-7319-4b0d-bd8d-fe20648264a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:05.950483 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26aa3c-7319-4b0d-bd8d-fe20648264a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:05.952399 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26aa3c-7319-4b0d-bd8d-fe20648264a5 None admin] RBAC: Adding query filter params (role.id=057d9335b3b1412ebfb6bfb27f02f96a, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=3e833722fe4e493e9ddc4b275632536f) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:05.952736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26aa3c-7319-4b0d-bd8d-fe20648264a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:05.954367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26aa3c-7319-4b0d-bd8d-fe20648264a5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:05.966214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 257/544] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:05 2022] GET /identity/v3/role_assignments?user.id=3e833722fe4e493e9ddc4b275632536f&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=057d9335b3b1412ebfb6bfb27f02f96a => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:05.966214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605daee-5ae7-450b-b207-69c7943230fc None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:05.966214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605daee-5ae7-450b-b207-69c7943230fc None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:05.966214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605daee-5ae7-450b-b207-69c7943230fc None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:05.966732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 258/545] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:05.975618 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6ba67-bd5f-44fe-a17e-c44d9fc5bb57 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:05.975934 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6ba67-bd5f-44fe-a17e-c44d9fc5bb57 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:05.976185 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6ba67-bd5f-44fe-a17e-c44d9fc5bb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:05.994958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-84d6ba67-bd5f-44fe-a17e-c44d9fc5bb57 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:05.996351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-84d6ba67-bd5f-44fe-a17e-c44d9fc5bb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:06.043740 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a622846-b080-4ce2-8f4f-97267b892d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:06.044387 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a622846-b080-4ce2-8f4f-97267b892d56 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:06.044674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a622846-b080-4ce2-8f4f-97267b892d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:06.044959 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a622846-b080-4ce2-8f4f-97267b892d56 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:06.045788 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a622846-b080-4ce2-8f4f-97267b892d56 None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:06.046130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a622846-b080-4ce2-8f4f-97267b892d56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:06.047579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a622846-b080-4ce2-8f4f-97267b892d56 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:06.068133 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 288/546] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:29:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:06.077853 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a622846-b080-4ce2-8f4f-97267b892d56 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:06.121190 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 259/547] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:06.172095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:06.181909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6ba67-bd5f-44fe-a17e-c44d9fc5bb57 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:06.225851 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a622846-b080-4ce2-8f4f-97267b892d56 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:06.249013 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:06.292357 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e48533-4afa-4d88-ae9e-4ec1198e9998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:06.293158 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e48533-4afa-4d88-ae9e-4ec1198e9998 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:06.293439 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e48533-4afa-4d88-ae9e-4ec1198e9998 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:06.293740 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e48533-4afa-4d88-ae9e-4ec1198e9998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:06.314512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e48533-4afa-4d88-ae9e-4ec1198e9998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:06.315992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e48533-4afa-4d88-ae9e-4ec1198e9998 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:06.354846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6ba67-bd5f-44fe-a17e-c44d9fc5bb57 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:06.358603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-44e48533-4afa-4d88-ae9e-4ec1198e9998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:06.358603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:06.358603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:06.358603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:06.358603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:06.358603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:06.358603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:06.358603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:06.358603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:06.358603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:06.358603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:06.358603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:06.358603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:06.358603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:06.358603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:06.359462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:06.359462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:06.359462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:06.359462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:06.359462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:06.359462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:06.359462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:06.359462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:06.359462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:06.359462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:06.359462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:06.359462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:06.359462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:06.359462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:06.359462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:06.359462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:06.359462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:06.360287 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:06.360287 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:06.360287 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:06.360287 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:06.362292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 289/548] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:06.367725 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e48533-4afa-4d88-ae9e-4ec1198e9998 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:06.427096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85724a5e-8b66-4bb5-bbbd-28f8ce45d2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:06.427905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85724a5e-8b66-4bb5-bbbd-28f8ce45d2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:06.428183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85724a5e-8b66-4bb5-bbbd-28f8ce45d2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:06.428457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85724a5e-8b66-4bb5-bbbd-28f8ce45d2c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:06.453665 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85724a5e-8b66-4bb5-bbbd-28f8ce45d2c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:06.467855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-85724a5e-8b66-4bb5-bbbd-28f8ce45d2c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:06.467855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:06.467855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:06.467855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:06.467855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:06.467855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:06.467855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:06.467855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:06.467855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:06.467855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:06.467855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:06.467855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:06.467855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:06.467855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:06.467855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:06.467855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:06.468886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:06.468886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:06.468886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:06.468886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:06.468886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:06.468886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:06.468886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:06.468886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:06.468886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:06.468886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:06.468886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:06.468886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:06.468886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:06.468886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:06.468886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:06.468886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:06.470260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:06.470260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:06.470260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:06.470260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:06.470260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:06.470260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:06.470260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:06.470260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:06.470260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:06.470260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:06.470260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:06.470260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:06.476536 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 260/549] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:06.479064 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85724a5e-8b66-4bb5-bbbd-28f8ce45d2c3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:06.594974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85724a5e-8b66-4bb5-bbbd-28f8ce45d2c3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:06.610318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:06.631124 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e48533-4afa-4d88-ae9e-4ec1198e9998 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:06.653592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ced7e-432e-44b1-9ac3-82ab9ef94b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:06.654291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ced7e-432e-44b1-9ac3-82ab9ef94b3e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:06.654571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ced7e-432e-44b1-9ac3-82ab9ef94b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:06.654845 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ced7e-432e-44b1-9ac3-82ab9ef94b3e None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:06.655587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ced7e-432e-44b1-9ac3-82ab9ef94b3e None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:06.655892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ced7e-432e-44b1-9ac3-82ab9ef94b3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:06.657280 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ced7e-432e-44b1-9ac3-82ab9ef94b3e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:06.662296 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:06.674446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 261/550] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:29:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:06.681154 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ced7e-432e-44b1-9ac3-82ab9ef94b3e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:06.695098 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c7ff9-8ef6-4bde-b145-ec64fc8d7376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:06.695763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c7ff9-8ef6-4bde-b145-ec64fc8d7376 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:06.696046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c7ff9-8ef6-4bde-b145-ec64fc8d7376 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:06.696320 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c7ff9-8ef6-4bde-b145-ec64fc8d7376 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:06.697307 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c7ff9-8ef6-4bde-b145-ec64fc8d7376 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:06.697623 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c7ff9-8ef6-4bde-b145-ec64fc8d7376 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:06.699667 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c7ff9-8ef6-4bde-b145-ec64fc8d7376 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:06.726144 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 290/551] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:06.730703 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c7ff9-8ef6-4bde-b145-ec64fc8d7376 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:06.822578 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ced7e-432e-44b1-9ac3-82ab9ef94b3e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:06.860196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb3490-4c0d-40e9-83f8-59aaff2ecb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:06.861124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb3490-4c0d-40e9-83f8-59aaff2ecb77 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:06.861407 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb3490-4c0d-40e9-83f8-59aaff2ecb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:06.861679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb3490-4c0d-40e9-83f8-59aaff2ecb77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:06.874166 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb3490-4c0d-40e9-83f8-59aaff2ecb77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:06.875779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb3490-4c0d-40e9-83f8-59aaff2ecb77 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:06.895007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-1afb3490-4c0d-40e9-83f8-59aaff2ecb77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:06.895007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:06.895007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:06.895007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:06.895007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:06.895007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:06.895007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:06.895007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:06.895007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:06.895007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:06.895007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:06.895007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:06.895007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:06.895007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:06.895007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:06.895993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:06.895993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:06.895993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:06.895993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:06.895993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:06.895993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:06.895993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:06.895993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:06.895993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:06.895993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:06.895993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:06.895993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:06.895993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:06.895993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:06.895993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:06.895993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:06.895993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:06.896930 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:06.896930 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:06.896930 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:06.896930 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:06.896930 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 262/552] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:06.904761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb3490-4c0d-40e9-83f8-59aaff2ecb77 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:06.950368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:06.994159 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c7ff9-8ef6-4bde-b145-ec64fc8d7376 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:07.016785 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb3490-4c0d-40e9-83f8-59aaff2ecb77 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:07.044691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ca120-ce7c-4553-a4c2-77cc88433b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:07.046133 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ca120-ce7c-4553-a4c2-77cc88433b79 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:07.046857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ca120-ce7c-4553-a4c2-77cc88433b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:07.047721 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ca120-ce7c-4553-a4c2-77cc88433b79 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:07.052219 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be38c51-d732-4d25-856d-c6dc43609f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:07.052883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be38c51-d732-4d25-856d-c6dc43609f3a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:07.053165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be38c51-d732-4d25-856d-c6dc43609f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:07.053438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be38c51-d732-4d25-856d-c6dc43609f3a None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:07.054189 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be38c51-d732-4d25-856d-c6dc43609f3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:07.054496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be38c51-d732-4d25-856d-c6dc43609f3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:07.055876 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be38c51-d732-4d25-856d-c6dc43609f3a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:07.057808 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ca120-ce7c-4553-a4c2-77cc88433b79 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:07.060747 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ca120-ce7c-4553-a4c2-77cc88433b79 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:07.069076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 263/553] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:07.070491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63261970-3dbd-416c-a9a9-dc96cb4797f6 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:07.070775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63261970-3dbd-416c-a9a9-dc96cb4797f6 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:07.071050 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63261970-3dbd-416c-a9a9-dc96cb4797f6 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:07.076409 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 264/554] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:07.077198 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63261970-3dbd-416c-a9a9-dc96cb4797f6 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:07.079399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-930ca120-ce7c-4553-a4c2-77cc88433b79 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:29:07.079399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:07.079399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:07.079399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:07.079399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:07.079399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:07.079399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:07.079399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:07.079399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:07.079399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:07.079399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:07.079399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:07.079399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:07.079399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:07.079399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:07.079399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:07.080395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:07.081300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:07.081300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:07.081300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:07.081300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:07.081300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:07.081300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:07.081300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:07.081300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:07.081300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:29:07.081300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:07.083423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 291/555] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:29:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:07.089099 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf084f-424f-46c1-95a2-50b549970cfb None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:07.089099 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf084f-424f-46c1-95a2-50b549970cfb None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:07.089099 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf084f-424f-46c1-95a2-50b549970cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:07.109168 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9abf084f-424f-46c1-95a2-50b549970cfb None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:07.110399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9abf084f-424f-46c1-95a2-50b549970cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:07.145365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:07.197487 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:07.202994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 292/556] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:07.203874 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf084f-424f-46c1-95a2-50b549970cfb None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:07.238702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63261970-3dbd-416c-a9a9-dc96cb4797f6 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:07.275053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b299f51-4d18-4095-9d21-97b1ff10940b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:07.275817 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b299f51-4d18-4095-9d21-97b1ff10940b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:07.276098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b299f51-4d18-4095-9d21-97b1ff10940b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:07.276375 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b299f51-4d18-4095-9d21-97b1ff10940b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:07.284690 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b299f51-4d18-4095-9d21-97b1ff10940b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:07.286228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b299f51-4d18-4095-9d21-97b1ff10940b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:07.303184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-2b299f51-4d18-4095-9d21-97b1ff10940b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:29:07.303184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:07.303184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:07.303184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:07.303184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:07.303184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:07.303184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:07.303184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:07.303184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:07.303184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:07.303184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:07.303184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:07.303184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:07.303184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:07.303184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:07.303184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:07.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:07.305615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:07.305615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:07.305615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:07.305615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:07.305615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:07.305615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:07.305615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:07.305615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:07.305615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:29:07.305615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:07.305615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 265/557] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jul 12 13:29:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:07.305615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b299f51-4d18-4095-9d21-97b1ff10940b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:07.330027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf084f-424f-46c1-95a2-50b549970cfb None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:07.378248 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59466d-2709-4f4f-93f5-1af17f09b559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:07.379794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59466d-2709-4f4f-93f5-1af17f09b559 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:07.380085 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59466d-2709-4f4f-93f5-1af17f09b559 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:07.380366 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59466d-2709-4f4f-93f5-1af17f09b559 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:07.381211 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59466d-2709-4f4f-93f5-1af17f09b559 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:07.381526 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59466d-2709-4f4f-93f5-1af17f09b559 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:07.383195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59466d-2709-4f4f-93f5-1af17f09b559 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:07.408378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 293/558] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Jul 12 13:29:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:07.409232 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59466d-2709-4f4f-93f5-1af17f09b559 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:07.410353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b299f51-4d18-4095-9d21-97b1ff10940b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:07.452489 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd86d4-f81d-4436-9137-9120e8a93722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:07.453333 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd86d4-f81d-4436-9137-9120e8a93722 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:07.453614 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd86d4-f81d-4436-9137-9120e8a93722 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:07.453891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd86d4-f81d-4436-9137-9120e8a93722 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:07.458441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:07.463884 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:07.469287 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd86d4-f81d-4436-9137-9120e8a93722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:07.479775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-70cd86d4-f81d-4436-9137-9120e8a93722 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:07.479775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:07.479775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:07.479775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:07.479775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:07.479775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:07.479775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:07.479775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:07.479775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:07.479775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:07.479775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:07.479775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:07.479775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:07.479775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:07.479775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:07.479775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:07.480894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:07.480894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:07.480894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:07.480894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:07.480894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:07.480894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:07.480894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:07.480894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:07.480894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:07.480894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:07.480894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:07.480894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:07.480894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:07.480894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:07.480894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:07.480894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:07.482425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:07.482425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:07.482425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:07.482425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:07.482425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:07.482425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:07.482425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:07.482425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:07.482425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:07.482425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:07.482425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:07.482425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:07.482425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 266/559] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:07.483098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd86d4-f81d-4436-9137-9120e8a93722 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:07.539765 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59466d-2709-4f4f-93f5-1af17f09b559 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:07.579719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2607c3-22cc-484b-99f9-59211fa744cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:07.580390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2607c3-22cc-484b-99f9-59211fa744cf None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:07.580676 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2607c3-22cc-484b-99f9-59211fa744cf None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:07.580952 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2607c3-22cc-484b-99f9-59211fa744cf None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:07.581791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2607c3-22cc-484b-99f9-59211fa744cf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:07.582134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2607c3-22cc-484b-99f9-59211fa744cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:07.583582 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2607c3-22cc-484b-99f9-59211fa744cf None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:07.604339 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 294/560] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jul 12 13:29:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:07.604621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd86d4-f81d-4436-9137-9120e8a93722 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:07.606906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2607c3-22cc-484b-99f9-59211fa744cf None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:07.643599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a710a77-5d0f-422b-946b-d4f91dcc61c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:07.644295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a710a77-5d0f-422b-946b-d4f91dcc61c1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:07.644583 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a710a77-5d0f-422b-946b-d4f91dcc61c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:07.645529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a710a77-5d0f-422b-946b-d4f91dcc61c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:07.696865 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:07.717735 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2607c3-22cc-484b-99f9-59211fa744cf None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:07.719472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:07.723071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a710a77-5d0f-422b-946b-d4f91dcc61c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:07.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a710a77-5d0f-422b-946b-d4f91dcc61c1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:07.753128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d37a3f-aab2-40c1-8c61-15b9b10b9258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:07.753795 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d37a3f-aab2-40c1-8c61-15b9b10b9258 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:07.754139 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d37a3f-aab2-40c1-8c61-15b9b10b9258 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:07.754423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d37a3f-aab2-40c1-8c61-15b9b10b9258 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:07.755174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d37a3f-aab2-40c1-8c61-15b9b10b9258 None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:07.755485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d37a3f-aab2-40c1-8c61-15b9b10b9258 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:07.756859 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d37a3f-aab2-40c1-8c61-15b9b10b9258 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:07.766600 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 295/561] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:29:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:07.767873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff707c2a-3b39-494a-a371-1000031d455c None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:07.768169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff707c2a-3b39-494a-a371-1000031d455c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:07.768447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff707c2a-3b39-494a-a371-1000031d455c None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:07.769830 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 296/562] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:07.774635 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff707c2a-3b39-494a-a371-1000031d455c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:07.840124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 267/563] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:29:07.841868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a710a77-5d0f-422b-946b-d4f91dcc61c1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:07.894272 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff707c2a-3b39-494a-a371-1000031d455c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:07.931397 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150f332-18f8-4ecf-bd63-25bfa26a8f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:07.932807 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150f332-18f8-4ecf-bd63-25bfa26a8f9f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:07.933171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150f332-18f8-4ecf-bd63-25bfa26a8f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:07.933524 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150f332-18f8-4ecf-bd63-25bfa26a8f9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:07.944698 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150f332-18f8-4ecf-bd63-25bfa26a8f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:07.946204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150f332-18f8-4ecf-bd63-25bfa26a8f9f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:07.948928 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:07.952601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:07.956526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a710a77-5d0f-422b-946b-d4f91dcc61c1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:07.966051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-0150f332-18f8-4ecf-bd63-25bfa26a8f9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:07.966051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:07.966051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:07.966051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:07.966051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:07.966051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:07.966051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:07.966051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:07.966051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:07.966051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:07.966051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:07.966051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:07.966051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:07.966051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:07.966051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:07.967136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:07.967136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:07.967136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:07.967136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:07.967136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:07.967136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:07.967136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:07.967136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:07.967136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:07.967136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:07.967136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:07.967136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:07.967136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:07.967136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:07.967136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:07.967136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:07.967136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:07.968069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:07.968069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:07.968069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:07.968069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:07.968069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 297/564] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:07.968794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93daf81e-1b5f-4a7d-ba2f-1829886d9ad0 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:07.969093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93daf81e-1b5f-4a7d-ba2f-1829886d9ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:07.969364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93daf81e-1b5f-4a7d-ba2f-1829886d9ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:07.987668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-93daf81e-1b5f-4a7d-ba2f-1829886d9ad0 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:07.988867 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-93daf81e-1b5f-4a7d-ba2f-1829886d9ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:08.000537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afa4b7-63a2-4e83-afe9-07b151bad3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:08.001332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afa4b7-63a2-4e83-afe9-07b151bad3ce None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:08.001616 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afa4b7-63a2-4e83-afe9-07b151bad3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:08.002195 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afa4b7-63a2-4e83-afe9-07b151bad3ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:08.015629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afa4b7-63a2-4e83-afe9-07b151bad3ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:08.017146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afa4b7-63a2-4e83-afe9-07b151bad3ce None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:08.035492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-55afa4b7-63a2-4e83-afe9-07b151bad3ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:08.035492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:08.035492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:08.035492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:08.035492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:08.035492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:08.035492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:08.035492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:08.035492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:08.035492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:08.035492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:08.035492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:08.035492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:08.035492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:08.035492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:08.036466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:08.036466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:08.036466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:08.036466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:08.036466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:08.036466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:08.036466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:08.036466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:08.036466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:08.036466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:08.036466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:08.036466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:08.036466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:08.036466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:08.036466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:08.036466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:08.036466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:08.042423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:08.042423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:08.042423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:08.042423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:08.042423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 268/565] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:08.042423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afa4b7-63a2-4e83-afe9-07b151bad3ce None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:08.072885 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 298/566] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:08.073878 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93daf81e-1b5f-4a7d-ba2f-1829886d9ad0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:08.154083 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afa4b7-63a2-4e83-afe9-07b151bad3ce None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:08.183601 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93daf81e-1b5f-4a7d-ba2f-1829886d9ad0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:08.190346 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2d27b-b776-468a-9456-ddd3c2418733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:08.191142 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2d27b-b776-468a-9456-ddd3c2418733 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:08.191430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2d27b-b776-468a-9456-ddd3c2418733 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:08.191708 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2d27b-b776-468a-9456-ddd3c2418733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:08.202626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2d27b-b776-468a-9456-ddd3c2418733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:08.204077 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2d27b-b776-468a-9456-ddd3c2418733 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:08.209856 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:08.219059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d5cfc-75b3-4b5f-8b96-6ef4e1deb82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:08.219721 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d5cfc-75b3-4b5f-8b96-6ef4e1deb82b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:08.220007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d5cfc-75b3-4b5f-8b96-6ef4e1deb82b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:08.220292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d5cfc-75b3-4b5f-8b96-6ef4e1deb82b None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:08.221013 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d5cfc-75b3-4b5f-8b96-6ef4e1deb82b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:08.221321 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d5cfc-75b3-4b5f-8b96-6ef4e1deb82b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:08.221598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-80e2d27b-b776-468a-9456-ddd3c2418733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:08.221598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:08.221598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:08.221598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:08.221598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:08.221598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:08.221598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:08.221598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:08.221598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:08.221598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:08.221598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:08.221598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:08.221598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:08.221598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:08.221598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:08.222501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:08.222501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:08.222501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:08.222501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:08.222501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:08.222501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:08.222501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:08.222501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:08.222501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:08.222501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:08.222501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:08.222501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:08.222501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:08.222501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:08.222501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:08.222501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:08.222501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:08.223403 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:08.223403 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:08.223403 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:08.223403 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:08.223618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d5cfc-75b3-4b5f-8b96-6ef4e1deb82b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:08.223687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 269/567] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:08.224123 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2d27b-b776-468a-9456-ddd3c2418733 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:08.228589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:08.237975 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 299/568] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:08.238812 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d5cfc-75b3-4b5f-8b96-6ef4e1deb82b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:08.324009 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2d27b-b776-468a-9456-ddd3c2418733 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:08.339780 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d5cfc-75b3-4b5f-8b96-6ef4e1deb82b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:08.358121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb01ec5-106c-4c17-b703-59021dbe17b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:08.358771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb01ec5-106c-4c17-b703-59021dbe17b3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:08.359049 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb01ec5-106c-4c17-b703-59021dbe17b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:08.359318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb01ec5-106c-4c17-b703-59021dbe17b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:08.360024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb01ec5-106c-4c17-b703-59021dbe17b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:08.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb01ec5-106c-4c17-b703-59021dbe17b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:08.361687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb01ec5-106c-4c17-b703-59021dbe17b3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:08.373331 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 270/569] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:08.374177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292d258-1d40-408e-b1c7-e80a0ad5e860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:08.374785 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe8759-55e0-4338-9272-a5149261b275 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:08.374974 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292d258-1d40-408e-b1c7-e80a0ad5e860 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:08.375085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe8759-55e0-4338-9272-a5149261b275 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:08.375258 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292d258-1d40-408e-b1c7-e80a0ad5e860 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:08.375356 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe8759-55e0-4338-9272-a5149261b275 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:08.375536 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292d258-1d40-408e-b1c7-e80a0ad5e860 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:08.380721 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 271/570] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:08.380721 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe8759-55e0-4338-9272-a5149261b275 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:08.386346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292d258-1d40-408e-b1c7-e80a0ad5e860 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:08.394226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-8292d258-1d40-408e-b1c7-e80a0ad5e860 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:08.394226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:08.394226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:08.394226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:08.394226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:08.394226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:08.394226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:08.394226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:08.394226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:08.394226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:08.394226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:08.394226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:08.394226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:08.394226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:08.394226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:08.394226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:08.395358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:08.395358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:08.395358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:08.395358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:08.395358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:08.395358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:08.395358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:08.395358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:08.395358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:08.395358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:08.395358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:08.395358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:08.395358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:08.395358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:08.395358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:08.395358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:08.396226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:08.396226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:08.396226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:08.396226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:08.396226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:08.396226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:08.396226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:08.396226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:08.396226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:08.396226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:08.396226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:08.396226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:08.396226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 300/571] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:08.396226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292d258-1d40-408e-b1c7-e80a0ad5e860 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:08.442658 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:08.447175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:08.485713 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe8759-55e0-4338-9272-a5149261b275 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:08.495964 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292d258-1d40-408e-b1c7-e80a0ad5e860 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:08.519316 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b5980-9c16-485f-9932-290a23488d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:08.519981 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b5980-9c16-485f-9932-290a23488d3c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:08.520263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b5980-9c16-485f-9932-290a23488d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:08.520542 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b5980-9c16-485f-9932-290a23488d3c None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:08.521266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b5980-9c16-485f-9932-290a23488d3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:08.521569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b5980-9c16-485f-9932-290a23488d3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:08.523006 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b5980-9c16-485f-9932-290a23488d3c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:08.530260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405e6ec-ff0d-49be-a897-73de37d2d418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:08.531054 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405e6ec-ff0d-49be-a897-73de37d2d418 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:08.531339 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405e6ec-ff0d-49be-a897-73de37d2d418 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:08.531615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405e6ec-ff0d-49be-a897-73de37d2d418 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:08.534799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 272/572] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:08.535593 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b5980-9c16-485f-9932-290a23488d3c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:08.538764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405e6ec-ff0d-49be-a897-73de37d2d418 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:08.540399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405e6ec-ff0d-49be-a897-73de37d2d418 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:08.556171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-f405e6ec-ff0d-49be-a897-73de37d2d418 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:08.556171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:08.556171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:08.556171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:08.556171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:08.556171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:08.556171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:08.556171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:08.556171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:08.556171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:08.556171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:08.556171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:08.556171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:08.556171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:08.556171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:08.557121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:08.557121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:08.557121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:08.557121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:08.557121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:08.557121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:08.557121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:08.557121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:08.557121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:08.557121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:08.557121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:08.557121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:08.557121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:08.557121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:08.557121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:08.557121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:08.557121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:08.558039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:08.558039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:08.558039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:08.558039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:08.558039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:08.558039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:08.558039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:08.558039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:08.558039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 301/573] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:08.558671 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df3d66-b058-4c62-9351-6cda5f7f71b0 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:08.558912 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df3d66-b058-4c62-9351-6cda5f7f71b0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:08.559214 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df3d66-b058-4c62-9351-6cda5f7f71b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:08.576182 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-68df3d66-b058-4c62-9351-6cda5f7f71b0 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:08.577397 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-68df3d66-b058-4c62-9351-6cda5f7f71b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:08.643487 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b5980-9c16-485f-9932-290a23488d3c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:08.644594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 302/574] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:08.645427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df3d66-b058-4c62-9351-6cda5f7f71b0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:08.681809 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:08.682082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:08.693868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116735a-2c3a-402a-a7cf-5ac290039d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:08.694608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116735a-2c3a-402a-a7cf-5ac290039d27 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:08.694896 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116735a-2c3a-402a-a7cf-5ac290039d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:08.695162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116735a-2c3a-402a-a7cf-5ac290039d27 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:08.702415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116735a-2c3a-402a-a7cf-5ac290039d27 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:08.703858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116735a-2c3a-402a-a7cf-5ac290039d27 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:08.719389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-d116735a-2c3a-402a-a7cf-5ac290039d27 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 13:29:08.719389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:08.719389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:08.719389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:08.719389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:08.719389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:08.719389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:08.719389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:08.719389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:08.719389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:08.719389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:08.719389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:08.719389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:08.719389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:08.719389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:08.719389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:08.720894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:08.721856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:08.721856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:08.721856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:08.721856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:08.721856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:08.721856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:08.721856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:08.721856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:08.721856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 13:29:08.721856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:08.721856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 273/575] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:29:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:08.721856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116735a-2c3a-402a-a7cf-5ac290039d27 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:08.750777 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df3d66-b058-4c62-9351-6cda5f7f71b0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:08.788588 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c4888-5b44-484a-9c5a-95c9ea9ee906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:08.789261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c4888-5b44-484a-9c5a-95c9ea9ee906 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:08.789544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c4888-5b44-484a-9c5a-95c9ea9ee906 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:08.789823 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c4888-5b44-484a-9c5a-95c9ea9ee906 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:08.790603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c4888-5b44-484a-9c5a-95c9ea9ee906 None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:08.790913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c4888-5b44-484a-9c5a-95c9ea9ee906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:08.792467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c4888-5b44-484a-9c5a-95c9ea9ee906 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:08.801751 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 303/576] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:29:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:08.802600 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c4888-5b44-484a-9c5a-95c9ea9ee906 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:08.824407 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116735a-2c3a-402a-a7cf-5ac290039d27 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:08.864894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117c778-0ff1-4135-9dd6-22a82262c68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:08.865699 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117c778-0ff1-4135-9dd6-22a82262c68c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:08.866008 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117c778-0ff1-4135-9dd6-22a82262c68c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:08.866291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117c778-0ff1-4135-9dd6-22a82262c68c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:08.873212 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117c778-0ff1-4135-9dd6-22a82262c68c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:08.874690 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117c778-0ff1-4135-9dd6-22a82262c68c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:08.888846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-4117c778-0ff1-4135-9dd6-22a82262c68c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:08.888846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:08.888846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:08.888846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:08.888846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:08.888846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:08.888846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:08.888846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:08.888846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:08.888846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:08.888846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:08.888846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:08.888846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:08.888846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:08.888846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:08.889924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:08.889924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:08.889924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:08.889924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:08.889924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:08.889924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:08.889924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:08.889924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:08.889924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:08.889924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:08.889924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:08.889924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:08.889924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:08.889924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:08.889924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:08.889924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:08.889924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:08.890965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:08.890965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:08.890965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:08.890965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:08.890965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:08.890965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:08.890965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:08.890965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:08.890965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 274/577] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:08.890965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117c778-0ff1-4135-9dd6-22a82262c68c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:08.905135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:08.911725 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c4888-5b44-484a-9c5a-95c9ea9ee906 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:08.940253 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:08.949835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a6a41-a64e-40e4-b7f8-b807f5f81fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:08.950428 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a6a41-a64e-40e4-b7f8-b807f5f81fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:08.950712 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a6a41-a64e-40e4-b7f8-b807f5f81fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:08.950987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a6a41-a64e-40e4-b7f8-b807f5f81fa8 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:08.951772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a6a41-a64e-40e4-b7f8-b807f5f81fa8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:08.952079 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a6a41-a64e-40e4-b7f8-b807f5f81fa8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:08.953534 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a6a41-a64e-40e4-b7f8-b807f5f81fa8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:08.968303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 304/578] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:08.968914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a6a41-a64e-40e4-b7f8-b807f5f81fa8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:09.003392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117c778-0ff1-4135-9dd6-22a82262c68c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:09.042103 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571dbdc-6e2d-49e5-bcf2-8f3d53cb8ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:09.042918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571dbdc-6e2d-49e5-bcf2-8f3d53cb8ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:09.043206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571dbdc-6e2d-49e5-bcf2-8f3d53cb8ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:09.043484 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571dbdc-6e2d-49e5-bcf2-8f3d53cb8ee4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:09.055468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571dbdc-6e2d-49e5-bcf2-8f3d53cb8ee4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:09.063463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-6571dbdc-6e2d-49e5-bcf2-8f3d53cb8ee4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:09.063463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:09.063463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:09.063463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:09.063463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:09.063463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:09.063463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:09.063463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:09.063463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:09.063463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:09.063463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:09.063463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:09.063463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:09.063463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:09.063463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:09.063463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:09.064529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:09.064529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:09.064529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:09.064529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:09.064529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:09.064529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:09.064529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:09.064529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:09.064529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:09.064529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:09.064529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:09.064529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:09.064529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:09.064529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:09.064529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:09.064529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:09.065359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:09.065359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:09.065359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:09.065359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:09.065359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:09.065359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:09.065359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:09.065359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:09.065359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:09.065359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:09.065359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:09.065359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:09.065359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 275/579] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:09.066082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571dbdc-6e2d-49e5-bcf2-8f3d53cb8ee4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:09.081799 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a6a41-a64e-40e4-b7f8-b807f5f81fa8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:09.120531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790540a-a475-4857-9bd9-6531a9336920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:09.121203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790540a-a475-4857-9bd9-6531a9336920 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:09.121484 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790540a-a475-4857-9bd9-6531a9336920 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:09.121757 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790540a-a475-4857-9bd9-6531a9336920 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:09.122631 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790540a-a475-4857-9bd9-6531a9336920 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:09.122938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790540a-a475-4857-9bd9-6531a9336920 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:09.124383 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790540a-a475-4857-9bd9-6531a9336920 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:09.144334 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 305/580] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Jul 12 13:29:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:09.145185 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790540a-a475-4857-9bd9-6531a9336920 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:09.164243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:09.174287 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571dbdc-6e2d-49e5-bcf2-8f3d53cb8ee4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:09.176939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:09.210158 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34fd67-9073-40f8-b597-d65b19c27784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:09.210868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34fd67-9073-40f8-b597-d65b19c27784 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:09.211151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34fd67-9073-40f8-b597-d65b19c27784 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:09.211422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34fd67-9073-40f8-b597-d65b19c27784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:09.223126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34fd67-9073-40f8-b597-d65b19c27784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:09.224588 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34fd67-9073-40f8-b597-d65b19c27784 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:09.242334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-9b34fd67-9073-40f8-b597-d65b19c27784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:09.242334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:09.242334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:09.242334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:09.242334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:09.242334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:09.242334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:09.242334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:09.242334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:09.242334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:09.242334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:09.242334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:09.242334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:09.242334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:09.242334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:09.243396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:09.243396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:09.243396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:09.243396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:09.243396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:09.243396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:09.243396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:09.243396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:09.243396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:09.243396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:09.243396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:09.243396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:09.243396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:09.243396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:09.243396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:09.243396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:09.243396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:09.244633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:09.244633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:09.244633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:09.244633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:09.244633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 276/581] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:09.244633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34fd67-9073-40f8-b597-d65b19c27784 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:09.255700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790540a-a475-4857-9bd9-6531a9336920 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:09.292473 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b35256-9105-4886-af15-b19b63aec22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:09.292862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b35256-9105-4886-af15-b19b63aec22b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:09.293153 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b35256-9105-4886-af15-b19b63aec22b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:09.293446 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b35256-9105-4886-af15-b19b63aec22b None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:09.294296 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b35256-9105-4886-af15-b19b63aec22b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:09.294612 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b35256-9105-4886-af15-b19b63aec22b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:09.296066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b35256-9105-4886-af15-b19b63aec22b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:09.310583 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 306/582] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:09.316725 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b35256-9105-4886-af15-b19b63aec22b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:09.354705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34fd67-9073-40f8-b597-d65b19c27784 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:09.390299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c47f17-bfa9-4b1d-885e-21f4f19efabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:09.391341 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c47f17-bfa9-4b1d-885e-21f4f19efabc None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:09.391635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c47f17-bfa9-4b1d-885e-21f4f19efabc None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:09.391948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c47f17-bfa9-4b1d-885e-21f4f19efabc None admin] PATH_INFO: `/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/0d3915cde0d04bcfb9b5ce8c89af1e5b/roles/057d9335b3b1412ebfb6bfb27f02f96a` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:09.400942 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:09.410917 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c47f17-bfa9-4b1d-885e-21f4f19efabc None admin] RBAC: Authorizing `identity:create_grant(project_id=36aa0326fd0545dface07a9ddc308ccd, user_id=0d3915cde0d04bcfb9b5ce8c89af1e5b, role_id=057d9335b3b1412ebfb6bfb27f02f96a)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:09.412409 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c47f17-bfa9-4b1d-885e-21f4f19efabc None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:09.427497 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 277/583] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:29:09 2022] PUT /identity/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/0d3915cde0d04bcfb9b5ce8c89af1e5b/roles/057d9335b3b1412ebfb6bfb27f02f96a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:09.428165 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b35256-9105-4886-af15-b19b63aec22b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:09.428348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c47f17-bfa9-4b1d-885e-21f4f19efabc None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:09.437703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:09.472625 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8e783-7c02-4e0a-bc15-4391ce481aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:09.473226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8e783-7c02-4e0a-bc15-4391ce481aea None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:09.473517 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8e783-7c02-4e0a-bc15-4391ce481aea None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:09.473796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8e783-7c02-4e0a-bc15-4391ce481aea None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:09.474608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a8e783-7c02-4e0a-bc15-4391ce481aea None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:09.474993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a8e783-7c02-4e0a-bc15-4391ce481aea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:09.476452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a8e783-7c02-4e0a-bc15-4391ce481aea None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:09.491993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 307/584] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:29:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:09.491993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8e783-7c02-4e0a-bc15-4391ce481aea None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:09.567395 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c47f17-bfa9-4b1d-885e-21f4f19efabc None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:09.620651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8e783-7c02-4e0a-bc15-4391ce481aea None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:09.627466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590d9e5-be01-4d7e-ba1e-b7f4a8038519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:09.628267 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590d9e5-be01-4d7e-ba1e-b7f4a8038519 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:09.628549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590d9e5-be01-4d7e-ba1e-b7f4a8038519 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:09.628822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590d9e5-be01-4d7e-ba1e-b7f4a8038519 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:09.640948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590d9e5-be01-4d7e-ba1e-b7f4a8038519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:09.642479 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590d9e5-be01-4d7e-ba1e-b7f4a8038519 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:09.661282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e77c3-a911-48d9-b934-92134c126792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:09.661912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-a590d9e5-be01-4d7e-ba1e-b7f4a8038519 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:09.661912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:09.661912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:09.661912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:09.661912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:09.661912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:09.661912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:09.661912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:09.661912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:09.661912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:09.661912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:09.661912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:09.661912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:09.661912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:09.661912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:09.662980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:09.662980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:09.662980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:09.662980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:09.662980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:09.662980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:09.662980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:09.662980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:09.662980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:09.662980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:09.662980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:09.662980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:09.662980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:09.662980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:09.662980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:09.662980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:09.662980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:09.664097 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:09.664097 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:09.664097 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:09.664097 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:09.664329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 278/585] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:09.664767 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e77c3-a911-48d9-b934-92134c126792 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:09.665292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e77c3-a911-48d9-b934-92134c126792 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:09.665700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e77c3-a911-48d9-b934-92134c126792 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:09.665796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590d9e5-be01-4d7e-ba1e-b7f4a8038519 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:09.666784 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e77c3-a911-48d9-b934-92134c126792 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:09.667102 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e77c3-a911-48d9-b934-92134c126792 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:09.668544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e77c3-a911-48d9-b934-92134c126792 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:09.678536 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:09.692065 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 308/586] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:09.692727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e77c3-a911-48d9-b934-92134c126792 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:09.732978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:09.788785 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590d9e5-be01-4d7e-ba1e-b7f4a8038519 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:09.828561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e77c3-a911-48d9-b934-92134c126792 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:09.829167 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7660b35-e300-4c6a-9950-c183a817b74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:09.829837 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7660b35-e300-4c6a-9950-c183a817b74d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:09.830131 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7660b35-e300-4c6a-9950-c183a817b74d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:09.830419 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7660b35-e300-4c6a-9950-c183a817b74d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:09.831252 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7660b35-e300-4c6a-9950-c183a817b74d None admin] RBAC: Adding query filter params (role.id=057d9335b3b1412ebfb6bfb27f02f96a, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=937bacab8c1c45c0876b8e61f6f3b859) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:09.831563 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7660b35-e300-4c6a-9950-c183a817b74d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:09.833083 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7660b35-e300-4c6a-9950-c183a817b74d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:09.840149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 279/587] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:09 2022] GET /identity/v3/role_assignments?user.id=937bacab8c1c45c0876b8e61f6f3b859&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=057d9335b3b1412ebfb6bfb27f02f96a => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:09.851494 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7660b35-e300-4c6a-9950-c183a817b74d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:09.905583 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3e335-5182-4242-a44b-85c773278c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:09.907268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3e335-5182-4242-a44b-85c773278c7d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:09.908687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3e335-5182-4242-a44b-85c773278c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:09.909262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3e335-5182-4242-a44b-85c773278c7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:09.932010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3e335-5182-4242-a44b-85c773278c7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:09.935405 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3e335-5182-4242-a44b-85c773278c7d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:09.972237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-81b3e335-5182-4242-a44b-85c773278c7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:09.972237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:09.972237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:09.972237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:09.972237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:09.972237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:09.972237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:09.972237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:09.972237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:09.972237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:09.972237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:09.972237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:09.972237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:09.972237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:09.972237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:09.973690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:09.973690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:09.973690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:09.973690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:09.973690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:09.973690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:09.973690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:09.973690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:09.973690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:09.973690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:09.973690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:09.973690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:09.973690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:09.973690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:09.973690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:09.973690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:09.973690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:09.975142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:09.975142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:09.975142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:09.975142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:09.979367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 309/588] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:09.981633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3e335-5182-4242-a44b-85c773278c7d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:10.046185 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7660b35-e300-4c6a-9950-c183a817b74d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:10.085019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:10.088329 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:10.107938 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26c80b-2b45-42f1-aba1-91306097bc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:10.108887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26c80b-2b45-42f1-aba1-91306097bc38 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:10.109330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26c80b-2b45-42f1-aba1-91306097bc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:10.109764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26c80b-2b45-42f1-aba1-91306097bc38 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:10.110825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26c80b-2b45-42f1-aba1-91306097bc38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:10.111295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26c80b-2b45-42f1-aba1-91306097bc38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:10.112862 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26c80b-2b45-42f1-aba1-91306097bc38 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:10.130670 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 280/589] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:10.138489 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26c80b-2b45-42f1-aba1-91306097bc38 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:10.205694 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3e335-5182-4242-a44b-85c773278c7d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:10.260501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f761eff-94de-4c82-8649-b845b8c4694b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:10.262306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f761eff-94de-4c82-8649-b845b8c4694b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:10.263050 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f761eff-94de-4c82-8649-b845b8c4694b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:10.263742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f761eff-94de-4c82-8649-b845b8c4694b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:10.272855 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f761eff-94de-4c82-8649-b845b8c4694b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:10.274681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f761eff-94de-4c82-8649-b845b8c4694b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:10.284176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26c80b-2b45-42f1-aba1-91306097bc38 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:10.291108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-0f761eff-94de-4c82-8649-b845b8c4694b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 13:29:10.291108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:10.291108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:10.291108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:10.291108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:10.291108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:10.291108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:10.291108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:10.291108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:10.291108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:10.291108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:10.291108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:10.291108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:10.291108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:10.291108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:10.291108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:10.292291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:10.293233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:10.293233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:10.293233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:10.293233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:10.293233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:10.293233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:10.293233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:10.293233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:10.293233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 13:29:10.293233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:10.296544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 310/590] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:29:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:10.297612 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f761eff-94de-4c82-8649-b845b8c4694b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:10.337222 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439714d5-2188-429d-9092-ac66cfa50783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:10.338325 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439714d5-2188-429d-9092-ac66cfa50783 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:10.338704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439714d5-2188-429d-9092-ac66cfa50783 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:10.339076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439714d5-2188-429d-9092-ac66cfa50783 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:10.341367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439714d5-2188-429d-9092-ac66cfa50783 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:10.341752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439714d5-2188-429d-9092-ac66cfa50783 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:10.343247 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439714d5-2188-429d-9092-ac66cfa50783 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:10.366021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 281/591] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:10.367219 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439714d5-2188-429d-9092-ac66cfa50783 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:10.421504 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:10.435858 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f761eff-94de-4c82-8649-b845b8c4694b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:10.451712 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:10.478904 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c3ef8-2c99-4cf0-aa47-3c506159ee6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:10.479896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c3ef8-2c99-4cf0-aa47-3c506159ee6d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:10.480278 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c3ef8-2c99-4cf0-aa47-3c506159ee6d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:10.480651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c3ef8-2c99-4cf0-aa47-3c506159ee6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:10.489218 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c3ef8-2c99-4cf0-aa47-3c506159ee6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:10.491191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c3ef8-2c99-4cf0-aa47-3c506159ee6d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:10.497830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439714d5-2188-429d-9092-ac66cfa50783 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:10.508066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-994c3ef8-2c99-4cf0-aa47-3c506159ee6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:10.508066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:10.508066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:10.508066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:10.508066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:10.508066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:10.508066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:10.508066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:10.508066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:10.508066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:10.508066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:10.508066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:10.508066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:10.508066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:10.508066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:10.509113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:10.509113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:10.509113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:10.509113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:10.509113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:10.509113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:10.509113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:10.509113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:10.509113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:10.509113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:10.509113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:10.509113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:10.509113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:10.509113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:10.509113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:10.509113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:10.509113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:10.510040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:10.510040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:10.510040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:10.510040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:10.510040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:10.510040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:10.510040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:10.510040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:10.515435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 311/592] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:10.516510 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c3ef8-2c99-4cf0-aa47-3c506159ee6d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:10.550075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd91f7-27d2-4049-a653-88b9db4b07cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:10.550075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd91f7-27d2-4049-a653-88b9db4b07cb None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:10.550075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd91f7-27d2-4049-a653-88b9db4b07cb None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:10.550075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd91f7-27d2-4049-a653-88b9db4b07cb None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:10.550468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd91f7-27d2-4049-a653-88b9db4b07cb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:10.550468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd91f7-27d2-4049-a653-88b9db4b07cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:10.551929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd91f7-27d2-4049-a653-88b9db4b07cb None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:10.580194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 282/593] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Jul 12 13:29:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:10.581634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd91f7-27d2-4049-a653-88b9db4b07cb None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:10.641526 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c3ef8-2c99-4cf0-aa47-3c506159ee6d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:10.682830 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57298f0-a21f-4d8f-a257-eb7b57f61cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:10.683787 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57298f0-a21f-4d8f-a257-eb7b57f61cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:10.684185 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57298f0-a21f-4d8f-a257-eb7b57f61cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:10.684527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57298f0-a21f-4d8f-a257-eb7b57f61cf0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:10.690925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:10.693651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57298f0-a21f-4d8f-a257-eb7b57f61cf0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:10.695096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd91f7-27d2-4049-a653-88b9db4b07cb None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:10.695872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57298f0-a21f-4d8f-a257-eb7b57f61cf0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:10.712186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-b57298f0-a21f-4d8f-a257-eb7b57f61cf0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 13:29:10.712186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:10.712186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:10.712186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:10.712186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:10.712186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:10.712186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:10.712186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:10.712186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:10.712186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:10.712186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:10.712186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:10.712186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:10.712186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:10.712186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:10.712186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:10.713426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:10.714587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:10.714587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:10.714587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:10.714587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:10.714587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:10.714587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:10.714587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:10.714587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:10.714587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 13:29:10.714587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:10.715119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 312/594] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:10.716288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57298f0-a21f-4d8f-a257-eb7b57f61cf0 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:10.738183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f0d41-e77b-44a8-9293-60e3146bb22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:10.739682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f0d41-e77b-44a8-9293-60e3146bb22c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:10.740147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f0d41-e77b-44a8-9293-60e3146bb22c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:10.740598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f0d41-e77b-44a8-9293-60e3146bb22c None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:10.741834 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f0d41-e77b-44a8-9293-60e3146bb22c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:10.742403 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f0d41-e77b-44a8-9293-60e3146bb22c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:10.744046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f0d41-e77b-44a8-9293-60e3146bb22c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:10.751571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:10.762015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 283/595] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:10.762015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f0d41-e77b-44a8-9293-60e3146bb22c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:10.836607 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57298f0-a21f-4d8f-a257-eb7b57f61cf0 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:10.871660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f0d41-e77b-44a8-9293-60e3146bb22c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:10.881359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6c666-1a5a-43f1-bbfb-8aea6aac1463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:10.882516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6c666-1a5a-43f1-bbfb-8aea6aac1463 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:10.883017 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6c666-1a5a-43f1-bbfb-8aea6aac1463 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:10.883466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6c666-1a5a-43f1-bbfb-8aea6aac1463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:10.898056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6c666-1a5a-43f1-bbfb-8aea6aac1463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:10.899992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6c666-1a5a-43f1-bbfb-8aea6aac1463 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:10.908913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a24a85-e4e1-4bfc-9874-41b49bca3e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:10.909589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a24a85-e4e1-4bfc-9874-41b49bca3e10 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:10.909871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a24a85-e4e1-4bfc-9874-41b49bca3e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:10.910173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a24a85-e4e1-4bfc-9874-41b49bca3e10 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:10.911011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a24a85-e4e1-4bfc-9874-41b49bca3e10 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:10.911320 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a24a85-e4e1-4bfc-9874-41b49bca3e10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:10.913017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a24a85-e4e1-4bfc-9874-41b49bca3e10 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:10.922791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-e6f6c666-1a5a-43f1-bbfb-8aea6aac1463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:10.922791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:10.922791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:10.922791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:10.922791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:10.922791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:10.922791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:10.922791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:10.922791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:10.922791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:10.922791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:10.922791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:10.922791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:10.922791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:10.922791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:10.923728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:10.923728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:10.923728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:10.923728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:10.923728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:10.923728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:10.923728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:10.923728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:10.923728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:10.923728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:10.923728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:10.923728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:10.923728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:10.923728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:10.923728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:10.923728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:10.923728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:10.924645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:10.924645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:10.924645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:10.924645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:10.926939 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 313/596] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:10.928052 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6c666-1a5a-43f1-bbfb-8aea6aac1463 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:10.931328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 284/597] 10.222.0.52 () {56 vars in 1139 bytes} [Tue Jul 12 13:29:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:10.932194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a24a85-e4e1-4bfc-9874-41b49bca3e10 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:10.953320 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:11.025915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:11.044553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a24a85-e4e1-4bfc-9874-41b49bca3e10 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:11.046120 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6c666-1a5a-43f1-bbfb-8aea6aac1463 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:11.081700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f6375-77a9-4ad2-a4f4-0401dd5f1c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:11.082556 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab6b51-5aeb-4ffa-a837-e050319eaa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:11.082865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f6375-77a9-4ad2-a4f4-0401dd5f1c7c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:11.083292 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f6375-77a9-4ad2-a4f4-0401dd5f1c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:11.083711 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f6375-77a9-4ad2-a4f4-0401dd5f1c7c None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:11.083823 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab6b51-5aeb-4ffa-a837-e050319eaa99 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:11.084159 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab6b51-5aeb-4ffa-a837-e050319eaa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:11.084524 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab6b51-5aeb-4ffa-a837-e050319eaa99 None admin] PATH_INFO: `/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/ebb4da7b00c64fafbb3724bae25b2853/roles/057d9335b3b1412ebfb6bfb27f02f96a` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:11.085068 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f6375-77a9-4ad2-a4f4-0401dd5f1c7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:11.085558 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f6375-77a9-4ad2-a4f4-0401dd5f1c7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:11.087222 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f6375-77a9-4ad2-a4f4-0401dd5f1c7c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:11.099656 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 285/598] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:11.100447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab6b51-5aeb-4ffa-a837-e050319eaa99 None admin] RBAC: Authorizing `identity:create_grant(project_id=36aa0326fd0545dface07a9ddc308ccd, user_id=ebb4da7b00c64fafbb3724bae25b2853, role_id=057d9335b3b1412ebfb6bfb27f02f96a)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:11.101347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f6375-77a9-4ad2-a4f4-0401dd5f1c7c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:11.102113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab6b51-5aeb-4ffa-a837-e050319eaa99 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:11.124133 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 314/599] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:29:10 2022] PUT /identity/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/ebb4da7b00c64fafbb3724bae25b2853/roles/057d9335b3b1412ebfb6bfb27f02f96a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:11.125211 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab6b51-5aeb-4ffa-a837-e050319eaa99 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:11.188631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:11.214413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f6375-77a9-4ad2-a4f4-0401dd5f1c7c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:11.257929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca58e0-8fdd-40d3-9073-9f8b700c23c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:11.261868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca58e0-8fdd-40d3-9073-9f8b700c23c4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:11.267782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca58e0-8fdd-40d3-9073-9f8b700c23c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:11.268410 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca58e0-8fdd-40d3-9073-9f8b700c23c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:11.289509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca58e0-8fdd-40d3-9073-9f8b700c23c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:11.292305 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca58e0-8fdd-40d3-9073-9f8b700c23c4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:11.319339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-fcca58e0-8fdd-40d3-9073-9f8b700c23c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:11.319339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:11.319339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:11.319339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:11.319339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:11.319339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:11.319339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:11.319339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:11.319339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:11.319339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:11.319339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:11.319339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:11.319339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:11.319339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:11.319339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:11.320291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:11.320291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:11.320291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:11.320291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:11.320291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:11.320291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:11.320291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:11.320291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:11.320291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:11.320291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:11.320291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:11.320291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:11.320291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:11.320291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:11.320291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:11.320291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:11.320291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:11.321180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:11.321180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:11.321180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:11.321180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:11.323695 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 286/600] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:11.331074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca58e0-8fdd-40d3-9073-9f8b700c23c4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:11.352845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab6b51-5aeb-4ffa-a837-e050319eaa99 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:11.424543 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:11.430874 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be66df-aa7e-4915-9366-a9230a2e7a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:11.434822 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be66df-aa7e-4915-9366-a9230a2e7a34 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:11.435360 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be66df-aa7e-4915-9366-a9230a2e7a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:11.437836 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be66df-aa7e-4915-9366-a9230a2e7a34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:11.455066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be66df-aa7e-4915-9366-a9230a2e7a34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:11.457142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be66df-aa7e-4915-9366-a9230a2e7a34 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:11.462997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca58e0-8fdd-40d3-9073-9f8b700c23c4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:11.476221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-77be66df-aa7e-4915-9366-a9230a2e7a34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:11.476221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:11.476221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:11.476221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:11.476221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:11.476221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:11.476221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:11.476221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:11.476221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:11.476221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:11.476221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:11.476221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:11.476221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:11.476221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:11.476221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:11.477172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:11.477172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:11.477172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:11.477172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:11.477172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:11.477172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:11.477172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:11.477172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:11.477172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:11.477172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:11.477172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:11.477172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:11.477172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:11.477172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:11.477172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:11.477172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:11.477172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:11.478069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:11.478069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:11.478069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:11.478069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:11.478069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:11.478069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:11.478069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:11.478069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:11.481563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 315/601] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:11.487942 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be66df-aa7e-4915-9366-a9230a2e7a34 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:11.509256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:11.527714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd65a60-11ac-48bf-9b95-1ba0166a316e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:11.528457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd65a60-11ac-48bf-9b95-1ba0166a316e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:11.528741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd65a60-11ac-48bf-9b95-1ba0166a316e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:11.529046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd65a60-11ac-48bf-9b95-1ba0166a316e None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:11.529765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd65a60-11ac-48bf-9b95-1ba0166a316e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:11.530100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd65a60-11ac-48bf-9b95-1ba0166a316e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:11.531512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd65a60-11ac-48bf-9b95-1ba0166a316e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:11.565833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 287/602] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:11.574071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd65a60-11ac-48bf-9b95-1ba0166a316e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:11.602816 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be66df-aa7e-4915-9366-a9230a2e7a34 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:11.641794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97465c25-da72-4a54-9d73-28923f62f7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:11.642838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97465c25-da72-4a54-9d73-28923f62f7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:11.643323 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97465c25-da72-4a54-9d73-28923f62f7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:11.643793 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97465c25-da72-4a54-9d73-28923f62f7f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:11.644792 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97465c25-da72-4a54-9d73-28923f62f7f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:11.645316 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97465c25-da72-4a54-9d73-28923f62f7f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:11.646992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97465c25-da72-4a54-9d73-28923f62f7f7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:11.660876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 316/603] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:11.667536 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97465c25-da72-4a54-9d73-28923f62f7f7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:11.754920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:11.783085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd65a60-11ac-48bf-9b95-1ba0166a316e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:11.822442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97465c25-da72-4a54-9d73-28923f62f7f7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:11.830951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e72e7b-6442-4662-81b2-f0a25d59c44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:11.831779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e72e7b-6442-4662-81b2-f0a25d59c44e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:11.832064 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e72e7b-6442-4662-81b2-f0a25d59c44e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:11.832337 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e72e7b-6442-4662-81b2-f0a25d59c44e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:11.842488 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e72e7b-6442-4662-81b2-f0a25d59c44e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:11.844316 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e72e7b-6442-4662-81b2-f0a25d59c44e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:11.865781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-63e72e7b-6442-4662-81b2-f0a25d59c44e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:11.865781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:11.865781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:11.865781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:11.865781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:11.865781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:11.865781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:11.865781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:11.865781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:11.865781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:11.865781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:11.865781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:11.865781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:11.865781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:11.865781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:11.866798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:11.866798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:11.866798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:11.866798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:11.866798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:11.866798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:11.866798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:11.866798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:11.866798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:11.866798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:11.866798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:11.866798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:11.866798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:11.866798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:11.866798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:11.866798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:11.866798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:11.867686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:11.867686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:11.867686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:11.867686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:11.867686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:11.867686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:11.867686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:11.867686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:11.867686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 288/604] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:11.868830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709627d2-bef3-475a-ac51-c0d2b60b6b67 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:11.869121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709627d2-bef3-475a-ac51-c0d2b60b6b67 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:11.869397 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709627d2-bef3-475a-ac51-c0d2b60b6b67 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:11.874976 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 289/605] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:11.874976 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709627d2-bef3-475a-ac51-c0d2b60b6b67 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:11.886597 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23522ce-5bb3-402a-81c7-0a4bbdb69109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:11.888048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23522ce-5bb3-402a-81c7-0a4bbdb69109 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:11.888550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23522ce-5bb3-402a-81c7-0a4bbdb69109 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:11.889028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23522ce-5bb3-402a-81c7-0a4bbdb69109 None admin] PATH_INFO: `/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/35336384bbca475082be349df406ae3f/roles/057d9335b3b1412ebfb6bfb27f02f96a` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:11.907550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23522ce-5bb3-402a-81c7-0a4bbdb69109 None admin] RBAC: Authorizing `identity:create_grant(project_id=36aa0326fd0545dface07a9ddc308ccd, user_id=35336384bbca475082be349df406ae3f, role_id=057d9335b3b1412ebfb6bfb27f02f96a)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:11.909251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23522ce-5bb3-402a-81c7-0a4bbdb69109 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:11.939007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 317/606] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:29:11 2022] PUT /identity/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/35336384bbca475082be349df406ae3f/roles/057d9335b3b1412ebfb6bfb27f02f96a => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:29:11.944537 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5411990-1417-48a6-a45b-52c0b7c1941f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:11.945204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5411990-1417-48a6-a45b-52c0b7c1941f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:11.945766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5411990-1417-48a6-a45b-52c0b7c1941f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:11.964312 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:11.987053 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c5411990-1417-48a6-a45b-52c0b7c1941f None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:11.988220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c5411990-1417-48a6-a45b-52c0b7c1941f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:12.047666 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709627d2-bef3-475a-ac51-c0d2b60b6b67 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:12.076139 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 318/607] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:12.082152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a59f0f-ca72-43ae-956a-ec4ba1481ed4 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:12.082152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a59f0f-ca72-43ae-956a-ec4ba1481ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:12.082152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a59f0f-ca72-43ae-956a-ec4ba1481ed4 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:12.084288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 319/608] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:12.085758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a157c4a-707e-4e75-9341-41875d873f89 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:12.086075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a157c4a-707e-4e75-9341-41875d873f89 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:12.086355 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a157c4a-707e-4e75-9341-41875d873f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:12.106472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1a157c4a-707e-4e75-9341-41875d873f89 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:12.108289 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1a157c4a-707e-4e75-9341-41875d873f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:12.115415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16579cd-3e41-41c0-bd5c-e94a24ecb15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:12.116438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16579cd-3e41-41c0-bd5c-e94a24ecb15d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:12.116951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16579cd-3e41-41c0-bd5c-e94a24ecb15d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:12.117443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16579cd-3e41-41c0-bd5c-e94a24ecb15d None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:12.118473 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16579cd-3e41-41c0-bd5c-e94a24ecb15d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:12.118993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16579cd-3e41-41c0-bd5c-e94a24ecb15d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:12.120666 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16579cd-3e41-41c0-bd5c-e94a24ecb15d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:12.135830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 290/609] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:12.136989 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16579cd-3e41-41c0-bd5c-e94a24ecb15d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:12.142948 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:12.204601 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 320/610] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:12.205412 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a157c4a-707e-4e75-9341-41875d873f89 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:12.283582 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16579cd-3e41-41c0-bd5c-e94a24ecb15d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:12.296347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:12.325608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb14638-4dc9-46b1-82af-9ddc3f7ca2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:12.326863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb14638-4dc9-46b1-82af-9ddc3f7ca2bd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:12.327347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb14638-4dc9-46b1-82af-9ddc3f7ca2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:12.327952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb14638-4dc9-46b1-82af-9ddc3f7ca2bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:12.341263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb14638-4dc9-46b1-82af-9ddc3f7ca2bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:12.350436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-bfb14638-4dc9-46b1-82af-9ddc3f7ca2bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:12.350436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:12.350436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:12.350436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:12.350436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:12.350436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:12.350436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:12.350436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:12.350436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:12.350436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:12.350436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:12.350436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:12.350436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:12.350436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:12.350436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:12.350436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:12.351469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:12.351469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:12.351469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:12.351469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:12.351469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:12.351469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:12.351469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:12.351469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:12.351469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:12.351469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:12.351469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:12.351469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:12.351469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:12.351469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:12.351469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:12.351469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:12.352299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:12.352299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:12.352299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:12.352299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:12.352299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:12.352299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:12.352299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:12.352299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:12.352299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:12.352299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:12.352299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:12.352299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:12.354671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 291/611] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:12.355609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb14638-4dc9-46b1-82af-9ddc3f7ca2bd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:12.358704 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a157c4a-707e-4e75-9341-41875d873f89 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:12.406349 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4053689-b489-4eb3-9459-d7eaddf90885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:12.407597 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4053689-b489-4eb3-9459-d7eaddf90885 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:12.407989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4053689-b489-4eb3-9459-d7eaddf90885 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:12.408370 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4053689-b489-4eb3-9459-d7eaddf90885 None admin] PATH_INFO: `/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/3e833722fe4e493e9ddc4b275632536f/roles/057d9335b3b1412ebfb6bfb27f02f96a` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:12.424938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4053689-b489-4eb3-9459-d7eaddf90885 None admin] RBAC: Authorizing `identity:create_grant(project_id=36aa0326fd0545dface07a9ddc308ccd, user_id=3e833722fe4e493e9ddc4b275632536f, role_id=057d9335b3b1412ebfb6bfb27f02f96a)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:12.426615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4053689-b489-4eb3-9459-d7eaddf90885 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:12.450176 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 321/612] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:29:12 2022] PUT /identity/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/3e833722fe4e493e9ddc4b275632536f/roles/057d9335b3b1412ebfb6bfb27f02f96a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:12.451551 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4053689-b489-4eb3-9459-d7eaddf90885 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:12.475702 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:12.506202 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb14638-4dc9-46b1-82af-9ddc3f7ca2bd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:12.552075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5a7ae-c37d-4e30-8aeb-fea3e3c11dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:12.552956 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5a7ae-c37d-4e30-8aeb-fea3e3c11dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:12.553239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5a7ae-c37d-4e30-8aeb-fea3e3c11dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:12.553520 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5a7ae-c37d-4e30-8aeb-fea3e3c11dd2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:12.580193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5a7ae-c37d-4e30-8aeb-fea3e3c11dd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:12.601329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-96c5a7ae-c37d-4e30-8aeb-fea3e3c11dd2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:12.601329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:12.601329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:12.601329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:12.601329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:12.601329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:12.601329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:12.601329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:12.601329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:12.601329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:12.601329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:12.601329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:12.601329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:12.601329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:12.601329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:12.601329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:12.603440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:12.603440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:12.603440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:12.603440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:12.603440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:12.603440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:12.603440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:12.603440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:12.603440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:12.603440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:12.603440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:12.603440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:12.603440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:12.603440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:12.603440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:12.603440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:12.605478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:12.605478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:12.605478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:12.605478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:12.605478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:12.605478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:12.605478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:12.605478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:12.605478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:12.605478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:12.605478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:12.605478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:12.605478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 292/613] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:12.613830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5a7ae-c37d-4e30-8aeb-fea3e3c11dd2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:12.686251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:12.688996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4053689-b489-4eb3-9459-d7eaddf90885 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:12.736482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e962f-37ee-48f4-8ca8-01c8fbf39e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:12.737112 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e962f-37ee-48f4-8ca8-01c8fbf39e23 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:12.737403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e962f-37ee-48f4-8ca8-01c8fbf39e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:12.738160 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e962f-37ee-48f4-8ca8-01c8fbf39e23 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:12.739266 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458e962f-37ee-48f4-8ca8-01c8fbf39e23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:12.739586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458e962f-37ee-48f4-8ca8-01c8fbf39e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:12.741192 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458e962f-37ee-48f4-8ca8-01c8fbf39e23 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:12.753382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 322/614] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:29:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:12.759726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e962f-37ee-48f4-8ca8-01c8fbf39e23 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:12.788566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5a7ae-c37d-4e30-8aeb-fea3e3c11dd2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:12.854598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e8dbc-bbcd-4aa5-af84-ce6dbabc91bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:12.854598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e8dbc-bbcd-4aa5-af84-ce6dbabc91bc None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:12.854598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e8dbc-bbcd-4aa5-af84-ce6dbabc91bc None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:12.854598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e8dbc-bbcd-4aa5-af84-ce6dbabc91bc None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:12.854974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e8dbc-bbcd-4aa5-af84-ce6dbabc91bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:12.854974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e8dbc-bbcd-4aa5-af84-ce6dbabc91bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:12.854974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e8dbc-bbcd-4aa5-af84-ce6dbabc91bc None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:12.869587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 293/615] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:29:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:12.879063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e8dbc-bbcd-4aa5-af84-ce6dbabc91bc None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:12.905915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:12.955312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e962f-37ee-48f4-8ca8-01c8fbf39e23 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:13.004764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:13.015557 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e8dbc-bbcd-4aa5-af84-ce6dbabc91bc None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:13.040110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4bdc2-741e-4cea-b7b9-e869540f2681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:13.040925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4bdc2-741e-4cea-b7b9-e869540f2681 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:13.041214 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4bdc2-741e-4cea-b7b9-e869540f2681 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:13.041493 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4bdc2-741e-4cea-b7b9-e869540f2681 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:13.058859 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d0425-7a27-4caa-8022-5d3183bc21a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:13.059732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d0425-7a27-4caa-8022-5d3183bc21a5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:13.060023 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d0425-7a27-4caa-8022-5d3183bc21a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:13.060300 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d0425-7a27-4caa-8022-5d3183bc21a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:13.068976 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4bdc2-741e-4cea-b7b9-e869540f2681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:13.070539 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4bdc2-741e-4cea-b7b9-e869540f2681 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:13.081173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d0425-7a27-4caa-8022-5d3183bc21a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:13.082952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d0425-7a27-4caa-8022-5d3183bc21a5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:13.106219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-a3e4bdc2-741e-4cea-b7b9-e869540f2681 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:13.106219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:13.106219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:13.106219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:13.106219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:13.106219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:13.106219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:13.106219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:13.106219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:13.106219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:13.106219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:13.106219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:13.106219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:13.106219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:13.106219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:13.107602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:13.107602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:13.107602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:13.107602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:13.107602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:13.107602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:13.107602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:13.107602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:13.107602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:13.107602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:13.107602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:13.107602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:13.107602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:13.107602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:13.107602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:13.107602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:13.107602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:13.108485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:13.108485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:13.108485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:13.108485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:13.108485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 323/616] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:13.113162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4bdc2-741e-4cea-b7b9-e869540f2681 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:13.118011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-411d0425-7a27-4caa-8022-5d3183bc21a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:13.118011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:13.118011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:13.118011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:13.118011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:13.118011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:13.118011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:13.118011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:13.118011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:13.118011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:13.118011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:13.118011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:13.118011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:13.118011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:13.118011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:13.118906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:13.118906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:13.118906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:13.118906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:13.118906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:13.118906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:13.118906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:13.118906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:13.118906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:13.118906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:13.118906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:13.118906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:13.118906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:13.118906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:13.118906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:13.118906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:13.118906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:13.119790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:13.119790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:13.119790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:13.119790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:13.120015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 294/617] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:13.126615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d0425-7a27-4caa-8022-5d3183bc21a5 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:13.239048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d0425-7a27-4caa-8022-5d3183bc21a5 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:13.280347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798234a-75f5-4dd3-9c89-33f17c6552ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:13.280941 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798234a-75f5-4dd3-9c89-33f17c6552ff None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:13.281289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798234a-75f5-4dd3-9c89-33f17c6552ff None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:13.281570 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798234a-75f5-4dd3-9c89-33f17c6552ff None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:13.285161 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798234a-75f5-4dd3-9c89-33f17c6552ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:13.285520 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798234a-75f5-4dd3-9c89-33f17c6552ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:13.286979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798234a-75f5-4dd3-9c89-33f17c6552ff None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:13.312924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 295/618] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:13.319276 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798234a-75f5-4dd3-9c89-33f17c6552ff None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:13.323782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:13.350570 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4bdc2-741e-4cea-b7b9-e869540f2681 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:13.402328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:13.422696 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9754314-97b6-45f8-b020-848cadc3b826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:13.423387 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9754314-97b6-45f8-b020-848cadc3b826 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:13.423670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9754314-97b6-45f8-b020-848cadc3b826 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:13.423948 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9754314-97b6-45f8-b020-848cadc3b826 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:13.424801 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9754314-97b6-45f8-b020-848cadc3b826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:13.425125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9754314-97b6-45f8-b020-848cadc3b826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:13.426969 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9754314-97b6-45f8-b020-848cadc3b826 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:13.457253 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798234a-75f5-4dd3-9c89-33f17c6552ff None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:13.463000 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 324/619] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:13.468102 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9754314-97b6-45f8-b020-848cadc3b826 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:13.494146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f7cae-3035-482f-a247-8746ea0a421b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:13.495126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f7cae-3035-482f-a247-8746ea0a421b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:13.495420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f7cae-3035-482f-a247-8746ea0a421b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:13.495700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f7cae-3035-482f-a247-8746ea0a421b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:13.517392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f7cae-3035-482f-a247-8746ea0a421b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:13.518975 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f7cae-3035-482f-a247-8746ea0a421b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:13.535015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-535f7cae-3035-482f-a247-8746ea0a421b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:29:13.535015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:13.535015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:13.535015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:13.535015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:13.535015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:13.535015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:13.535015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:13.535015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:13.535015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:13.535015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:13.535015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:13.535015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:13.535015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:13.535015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:13.535015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:13.536086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:13.537029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:13.537029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:13.537029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:13.537029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:13.537029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:13.537029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:13.537029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:13.537029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:13.537029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:29:13.537029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:13.537029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 296/620] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:29:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:13.542197 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f7cae-3035-482f-a247-8746ea0a421b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:13.613275 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9754314-97b6-45f8-b020-848cadc3b826 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:13.653825 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec414c-df22-457f-8c73-c46af0c5f21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:13.654653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec414c-df22-457f-8c73-c46af0c5f21f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:13.654934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec414c-df22-457f-8c73-c46af0c5f21f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:13.655210 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec414c-df22-457f-8c73-c46af0c5f21f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:13.666972 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:13.667255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec414c-df22-457f-8c73-c46af0c5f21f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:13.669761 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec414c-df22-457f-8c73-c46af0c5f21f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:13.700294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-69ec414c-df22-457f-8c73-c46af0c5f21f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:29:13.700294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:13.700294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:13.700294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:13.700294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:13.700294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:13.700294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:13.700294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:13.700294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:13.700294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:13.700294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:13.700294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:13.700294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:13.700294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:13.700294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:13.700294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:13.702685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:13.705038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:13.705038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:13.705038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:13.705038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:13.705038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:13.705038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:13.705038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:13.705038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:13.705038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:29:13.705038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:13.705038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 325/621] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jul 12 13:29:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:13.707750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec414c-df22-457f-8c73-c46af0c5f21f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:13.736548 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f7cae-3035-482f-a247-8746ea0a421b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:13.789628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8abc9b-0d39-4a49-ab0e-8a243de18487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:13.790259 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8abc9b-0d39-4a49-ab0e-8a243de18487 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:13.790508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8abc9b-0d39-4a49-ab0e-8a243de18487 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:13.790780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8abc9b-0d39-4a49-ab0e-8a243de18487 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:13.791629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8abc9b-0d39-4a49-ab0e-8a243de18487 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:13.791936 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8abc9b-0d39-4a49-ab0e-8a243de18487 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:13.793951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8abc9b-0d39-4a49-ab0e-8a243de18487 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:13.804764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:13.831661 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 297/622] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Jul 12 13:29:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:13.833844 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2f052-1830-4dac-af87-cfecbf79392b None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:13.834168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2f052-1830-4dac-af87-cfecbf79392b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:13.834446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2f052-1830-4dac-af87-cfecbf79392b None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:13.836107 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 298/623] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:13.837194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2f052-1830-4dac-af87-cfecbf79392b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:13.887837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec414c-df22-457f-8c73-c46af0c5f21f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:13.933410 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1f279-bd14-4486-b815-910165e13fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:13.934149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1f279-bd14-4486-b815-910165e13fae None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:13.934439 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1f279-bd14-4486-b815-910165e13fae None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:13.934717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1f279-bd14-4486-b815-910165e13fae None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:13.935576 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1f279-bd14-4486-b815-910165e13fae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:13.935890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1f279-bd14-4486-b815-910165e13fae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:13.937441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1f279-bd14-4486-b815-910165e13fae None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:13.963845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 326/624] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jul 12 13:29:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:13.966006 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eb749-9f9a-4d0a-a21e-035372d7eabc None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:13.966353 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eb749-9f9a-4d0a-a21e-035372d7eabc None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:13.966636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eb749-9f9a-4d0a-a21e-035372d7eabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:13.988998 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-aa6eb749-9f9a-4d0a-a21e-035372d7eabc None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:13.990329 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-aa6eb749-9f9a-4d0a-a21e-035372d7eabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:13.990960 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2f052-1830-4dac-af87-cfecbf79392b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:14.031422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:14.048468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df29cd-6ae9-4ae6-8c37-95ac3ceb3db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:14.049244 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df29cd-6ae9-4ae6-8c37-95ac3ceb3db7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:14.049531 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df29cd-6ae9-4ae6-8c37-95ac3ceb3db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:14.049808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df29cd-6ae9-4ae6-8c37-95ac3ceb3db7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:14.064102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df29cd-6ae9-4ae6-8c37-95ac3ceb3db7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:14.065887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df29cd-6ae9-4ae6-8c37-95ac3ceb3db7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:14.086410 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 327/625] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:14.087367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-a9df29cd-6ae9-4ae6-8c37-95ac3ceb3db7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:14.087367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:14.087367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:14.087367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:14.087367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:14.087367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:14.087367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:14.087367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:14.087367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:14.087367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:14.087367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:14.087367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:14.087367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:14.087367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:14.087367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:14.088277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:14.088277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:14.088277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:14.088277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:14.088277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:14.088277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:14.088277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:14.088277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:14.088277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:14.088277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:14.088277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:14.088277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:14.088277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:14.088277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:14.088277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:14.088277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:14.088277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:14.089201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:14.089201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:14.089201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:14.089201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:14.089449 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6eb749-9f9a-4d0a-a21e-035372d7eabc None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:14.089525 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 299/626] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:14.098975 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df29cd-6ae9-4ae6-8c37-95ac3ceb3db7 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:14.137972 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:14.213208 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df29cd-6ae9-4ae6-8c37-95ac3ceb3db7 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:14.229344 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6eb749-9f9a-4d0a-a21e-035372d7eabc None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:14.250965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc8f25-fe77-465d-b31b-4bb466aeb733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:14.251914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc8f25-fe77-465d-b31b-4bb466aeb733 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:14.252209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc8f25-fe77-465d-b31b-4bb466aeb733 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:14.252486 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc8f25-fe77-465d-b31b-4bb466aeb733 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:14.253224 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc8f25-fe77-465d-b31b-4bb466aeb733 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:14.253538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc8f25-fe77-465d-b31b-4bb466aeb733 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:14.254982 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc8f25-fe77-465d-b31b-4bb466aeb733 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:14.269456 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c15360-94e5-468f-8c22-270830b8da82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:14.270146 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c15360-94e5-468f-8c22-270830b8da82 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:14.270605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c15360-94e5-468f-8c22-270830b8da82 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:14.270889 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c15360-94e5-468f-8c22-270830b8da82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:14.272816 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 300/627] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:14.273809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc8f25-fe77-465d-b31b-4bb466aeb733 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:14.310883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:14.352534 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c15360-94e5-468f-8c22-270830b8da82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:14.353234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c15360-94e5-468f-8c22-270830b8da82 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:14.384104 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc8f25-fe77-465d-b31b-4bb466aeb733 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:14.389298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:14.424627 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5373c2-f146-4846-893f-69f944402dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:14.425462 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5373c2-f146-4846-893f-69f944402dcd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:14.425740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5373c2-f146-4846-893f-69f944402dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:14.426040 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5373c2-f146-4846-893f-69f944402dcd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:14.437954 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5373c2-f146-4846-893f-69f944402dcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:14.448180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-8c5373c2-f146-4846-893f-69f944402dcd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:14.448180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:14.448180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:14.448180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:14.448180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:14.448180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:14.448180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:14.448180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:14.448180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:14.448180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:14.448180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:14.448180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:14.448180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:14.448180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:14.448180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:14.448180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:14.449437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:14.449437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:14.449437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:14.449437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:14.449437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:14.449437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:14.449437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:14.449437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:14.449437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:14.449437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:14.449437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:14.449437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:14.449437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:14.449437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:14.449437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:14.449437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:14.450997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:14.450997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:14.450997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:14.450997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:14.450997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:14.450997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:14.450997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:14.450997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:14.450997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:14.450997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:14.450997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:14.450997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:14.452392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 301/628] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:14.453257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5373c2-f146-4846-893f-69f944402dcd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:14.470148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 328/629] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:29:14.471782 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ee328-e5be-4a2b-9dc2-146826b46b56 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:14.472136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ee328-e5be-4a2b-9dc2-146826b46b56 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:14.472468 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ee328-e5be-4a2b-9dc2-146826b46b56 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:14.477862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 329/630] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:14.477862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ee328-e5be-4a2b-9dc2-146826b46b56 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:14.571873 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:14.584180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5373c2-f146-4846-893f-69f944402dcd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:14.611577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ee328-e5be-4a2b-9dc2-146826b46b56 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:14.631307 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20692f61-95ec-4ee9-b6b5-de8014e499e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:14.632458 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20692f61-95ec-4ee9-b6b5-de8014e499e7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:14.632896 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20692f61-95ec-4ee9-b6b5-de8014e499e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:14.633367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20692f61-95ec-4ee9-b6b5-de8014e499e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:14.642862 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20692f61-95ec-4ee9-b6b5-de8014e499e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:14.644934 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20692f61-95ec-4ee9-b6b5-de8014e499e7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:14.653916 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d213800-f2e0-4bf9-a723-0191d2eb21fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:14.654623 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d213800-f2e0-4bf9-a723-0191d2eb21fd None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:14.654907 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d213800-f2e0-4bf9-a723-0191d2eb21fd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:14.655183 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d213800-f2e0-4bf9-a723-0191d2eb21fd None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:14.655940 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d213800-f2e0-4bf9-a723-0191d2eb21fd None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:14.656250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d213800-f2e0-4bf9-a723-0191d2eb21fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:14.657638 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d213800-f2e0-4bf9-a723-0191d2eb21fd None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:14.659206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:14.663421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-20692f61-95ec-4ee9-b6b5-de8014e499e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:14.663421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:14.663421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:14.663421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:14.663421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:14.663421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:14.663421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:14.663421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:14.663421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:14.663421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:14.663421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:14.663421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:14.663421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:14.663421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:14.663421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:14.664448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:14.664448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:14.664448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:14.664448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:14.664448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:14.664448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:14.664448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:14.664448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:14.664448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:14.664448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:14.664448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:14.664448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:14.664448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:14.664448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:14.664448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:14.664448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:14.664448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:14.665421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:14.665421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:14.665421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:14.665421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:14.665421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:14.665421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:14.665421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:14.665421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:14.665421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 302/631] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:14.665948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20692f61-95ec-4ee9-b6b5-de8014e499e7 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:14.677553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 330/632] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:29:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:14.677553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46004df-875a-40c3-ad99-03f1cabd1174 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:14.677553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46004df-875a-40c3-ad99-03f1cabd1174 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:14.677553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46004df-875a-40c3-ad99-03f1cabd1174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:14.705646 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b46004df-875a-40c3-ad99-03f1cabd1174 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:14.706872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b46004df-875a-40c3-ad99-03f1cabd1174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:14.782014 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 331/633] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:14.786138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20692f61-95ec-4ee9-b6b5-de8014e499e7 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:14.790897 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46004df-875a-40c3-ad99-03f1cabd1174 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:14.826871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349fd56-a77f-4bdb-889f-478dd53d3367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:14.827588 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349fd56-a77f-4bdb-889f-478dd53d3367 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:14.827872 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349fd56-a77f-4bdb-889f-478dd53d3367 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:14.828143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349fd56-a77f-4bdb-889f-478dd53d3367 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:14.839525 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349fd56-a77f-4bdb-889f-478dd53d3367 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:14.841071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349fd56-a77f-4bdb-889f-478dd53d3367 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:14.866915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-0349fd56-a77f-4bdb-889f-478dd53d3367 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:14.866915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:14.866915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:14.866915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:14.866915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:14.866915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:14.866915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:14.866915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:14.866915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:14.866915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:14.866915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:14.866915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:14.866915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:14.866915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:14.866915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:14.867899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:14.867899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:14.867899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:14.867899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:14.867899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:14.867899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:14.867899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:14.867899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:14.867899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:14.867899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:14.867899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:14.867899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:14.867899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:14.867899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:14.867899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:14.867899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:14.867899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:14.868782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:14.868782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:14.868782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:14.868782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:14.868782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 303/634] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:14.869071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349fd56-a77f-4bdb-889f-478dd53d3367 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:14.879315 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:14.920401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46004df-875a-40c3-ad99-03f1cabd1174 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:14.947887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:14.958390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc4a76-b673-449a-adc8-7a7f5e707707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:14.959066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc4a76-b673-449a-adc8-7a7f5e707707 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:14.959360 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc4a76-b673-449a-adc8-7a7f5e707707 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:14.959633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc4a76-b673-449a-adc8-7a7f5e707707 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:14.960431 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc4a76-b673-449a-adc8-7a7f5e707707 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:14.960743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc4a76-b673-449a-adc8-7a7f5e707707 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:14.962360 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc4a76-b673-449a-adc8-7a7f5e707707 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:14.978423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 332/635] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:14.979276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc4a76-b673-449a-adc8-7a7f5e707707 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:14.997285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349fd56-a77f-4bdb-889f-478dd53d3367 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.034432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4c101-5566-42a1-99cf-393dc2bb5efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.035250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4c101-5566-42a1-99cf-393dc2bb5efd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.035534 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4c101-5566-42a1-99cf-393dc2bb5efd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.035823 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4c101-5566-42a1-99cf-393dc2bb5efd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.047479 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4c101-5566-42a1-99cf-393dc2bb5efd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:15.048962 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4c101-5566-42a1-99cf-393dc2bb5efd None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:15.068242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-7ff4c101-5566-42a1-99cf-393dc2bb5efd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:15.068242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:15.068242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:15.068242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:15.068242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:15.068242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:15.068242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:15.068242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:15.068242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:15.068242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:15.068242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:15.068242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:15.068242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:15.068242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:15.068242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:15.069432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:15.069432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:15.069432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:15.069432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:15.069432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:15.069432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:15.069432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:15.069432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:15.069432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:15.069432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:15.069432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:15.069432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:15.069432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:15.069432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:15.069432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:15.069432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:15.069432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:15.070592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:15.070592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:15.070592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:15.070592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:15.070592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 304/636] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:15.070888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4c101-5566-42a1-99cf-393dc2bb5efd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:15.102838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc4a76-b673-449a-adc8-7a7f5e707707 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.135213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:15.141799 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6249c7-dba9-4322-8552-e284a9482a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.142627 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6249c7-dba9-4322-8552-e284a9482a32 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.142915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6249c7-dba9-4322-8552-e284a9482a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.143198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6249c7-dba9-4322-8552-e284a9482a32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.153876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6249c7-dba9-4322-8552-e284a9482a32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:15.161728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-7f6249c7-dba9-4322-8552-e284a9482a32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:15.161728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:15.161728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:15.161728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:15.161728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:15.161728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:15.161728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:15.161728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:15.161728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:15.161728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:15.161728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:15.161728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:15.161728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:15.161728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:15.161728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:15.161728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:15.162993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:15.162993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:15.162993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:15.162993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:15.162993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:15.162993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:15.162993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:15.162993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:15.162993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:15.162993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:15.162993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:15.162993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:15.162993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:15.162993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:15.162993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:15.162993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:15.164077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:15.164077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:15.164077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:15.164077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:15.164077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:15.164077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:15.164077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:15.164077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:15.164077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:15.164077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:15.164077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:15.164077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:15.164077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 333/637] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:15.164935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7af1b0-c19c-4310-9f83-b9b9f6717241 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.164935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7af1b0-c19c-4310-9f83-b9b9f6717241 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.165191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7af1b0-c19c-4310-9f83-b9b9f6717241 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.170224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 334/638] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:15.171066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7af1b0-c19c-4310-9f83-b9b9f6717241 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:15.181123 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4c101-5566-42a1-99cf-393dc2bb5efd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.215254 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67584bab-cca3-4a24-86e6-565799e988e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.215917 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67584bab-cca3-4a24-86e6-565799e988e1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.216202 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67584bab-cca3-4a24-86e6-565799e988e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.216471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67584bab-cca3-4a24-86e6-565799e988e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.217188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67584bab-cca3-4a24-86e6-565799e988e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:15.217485 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67584bab-cca3-4a24-86e6-565799e988e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:15.218882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67584bab-cca3-4a24-86e6-565799e988e1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:15.221185 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:15.230536 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 305/639] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:15.232262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67584bab-cca3-4a24-86e6-565799e988e1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:15.277227 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7af1b0-c19c-4310-9f83-b9b9f6717241 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.313071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9e6ac-a8ba-4482-ac27-de34d6cb1a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.313983 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9e6ac-a8ba-4482-ac27-de34d6cb1a78 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.314261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9e6ac-a8ba-4482-ac27-de34d6cb1a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.314528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9e6ac-a8ba-4482-ac27-de34d6cb1a78 None admin] PATH_INFO: `/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/937bacab8c1c45c0876b8e61f6f3b859/roles/057d9335b3b1412ebfb6bfb27f02f96a` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.329507 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced9e6ac-a8ba-4482-ac27-de34d6cb1a78 None admin] RBAC: Authorizing `identity:create_grant(project_id=36aa0326fd0545dface07a9ddc308ccd, user_id=937bacab8c1c45c0876b8e61f6f3b859, role_id=057d9335b3b1412ebfb6bfb27f02f96a)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:15.331058 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced9e6ac-a8ba-4482-ac27-de34d6cb1a78 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:15.337418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67584bab-cca3-4a24-86e6-565799e988e1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.345121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 335/640] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:29:15 2022] PUT /identity/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/937bacab8c1c45c0876b8e61f6f3b859/roles/057d9335b3b1412ebfb6bfb27f02f96a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:15.346498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9e6ac-a8ba-4482-ac27-de34d6cb1a78 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:15.357806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:15.379285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c75d61-e025-40ff-857f-3eb1a0a6f900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.380123 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c75d61-e025-40ff-857f-3eb1a0a6f900 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.380740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c75d61-e025-40ff-857f-3eb1a0a6f900 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.380740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c75d61-e025-40ff-857f-3eb1a0a6f900 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.381407 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c75d61-e025-40ff-857f-3eb1a0a6f900 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:15.381713 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c75d61-e025-40ff-857f-3eb1a0a6f900 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:15.383177 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c75d61-e025-40ff-857f-3eb1a0a6f900 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:15.397689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 306/641] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:15.404057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb1513-4448-4544-a375-65862e4aec6b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.404405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb1513-4448-4544-a375-65862e4aec6b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.404628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb1513-4448-4544-a375-65862e4aec6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.443264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7cfb1513-4448-4544-a375-65862e4aec6b None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:15.445461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7cfb1513-4448-4544-a375-65862e4aec6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:15.486265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:15.499289 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9e6ac-a8ba-4482-ac27-de34d6cb1a78 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.550485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f33b84-0787-47d5-b6c1-2f75df3f3f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.551162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f33b84-0787-47d5-b6c1-2f75df3f3f09 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.551444 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f33b84-0787-47d5-b6c1-2f75df3f3f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.551717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f33b84-0787-47d5-b6c1-2f75df3f3f09 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.552464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f33b84-0787-47d5-b6c1-2f75df3f3f09 None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:15.552772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f33b84-0787-47d5-b6c1-2f75df3f3f09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:15.554524 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f33b84-0787-47d5-b6c1-2f75df3f3f09 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:15.564587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 336/642] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:29:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:15.565218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 307/643] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:15.567046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb1513-4448-4544-a375-65862e4aec6b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:15.569753 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f33b84-0787-47d5-b6c1-2f75df3f3f09 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:15.701041 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb1513-4448-4544-a375-65862e4aec6b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.712706 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:15.717839 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f33b84-0787-47d5-b6c1-2f75df3f3f09 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.752335 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b60d2-14c2-49b2-9ccb-9ff7277674dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.753177 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b60d2-14c2-49b2-9ccb-9ff7277674dc None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.753452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b60d2-14c2-49b2-9ccb-9ff7277674dc None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.753720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b60d2-14c2-49b2-9ccb-9ff7277674dc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.759481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc642a-1079-4fba-b255-57a1f2f2efb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.760293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc642a-1079-4fba-b255-57a1f2f2efb1 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.760583 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc642a-1079-4fba-b255-57a1f2f2efb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.760850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc642a-1079-4fba-b255-57a1f2f2efb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.762717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b60d2-14c2-49b2-9ccb-9ff7277674dc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:15.764237 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b60d2-14c2-49b2-9ccb-9ff7277674dc None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:15.771246 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc642a-1079-4fba-b255-57a1f2f2efb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:15.774327 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc642a-1079-4fba-b255-57a1f2f2efb1 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:15.782428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-421b60d2-14c2-49b2-9ccb-9ff7277674dc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 13:29:15.782428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:15.782428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:15.782428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:15.782428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:15.782428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:15.782428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:15.782428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:15.782428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:15.782428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:15.782428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:15.782428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:15.782428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:15.782428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:15.782428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:15.782428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:15.783509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:15.784444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:15.784444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:15.784444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:15.784444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:15.784444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:15.784444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:15.784444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:15.784444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:15.784444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 13:29:15.784444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:15.784444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 308/644] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:29:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:15.785550 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b60d2-14c2-49b2-9ccb-9ff7277674dc None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:15.797650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-c7bc642a-1079-4fba-b255-57a1f2f2efb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:15.797650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:15.797650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:15.797650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:15.797650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:15.797650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:15.797650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:15.797650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:15.797650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:15.797650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:15.797650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:15.797650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:15.797650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:15.797650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:15.797650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:15.798700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:15.798700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:15.798700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:15.798700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:15.798700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:15.798700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:15.798700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:15.798700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:15.798700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:15.798700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:15.798700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:15.798700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:15.798700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:15.798700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:15.798700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:15.798700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:15.798700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:15.799729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:15.799729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:15.799729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:15.799729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:15.799729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:15.799729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:15.799729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:15.799729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:15.800174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 337/645] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:15.800894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc642a-1079-4fba-b255-57a1f2f2efb1 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:15.828518 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:15.927660 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc642a-1079-4fba-b255-57a1f2f2efb1 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.932041 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b60d2-14c2-49b2-9ccb-9ff7277674dc None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.975668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5d35a-080f-4f8a-aae1-a586ee00b830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.976468 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5d35a-080f-4f8a-aae1-a586ee00b830 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.976686 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5d35a-080f-4f8a-aae1-a586ee00b830 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.976959 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5d35a-080f-4f8a-aae1-a586ee00b830 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.984871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe291f8-2529-42e5-8b4f-3799ced9655c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.985675 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe291f8-2529-42e5-8b4f-3799ced9655c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.986019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe291f8-2529-42e5-8b4f-3799ced9655c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.986274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe291f8-2529-42e5-8b4f-3799ced9655c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.995200 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5d35a-080f-4f8a-aae1-a586ee00b830 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:16.002623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe291f8-2529-42e5-8b4f-3799ced9655c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:16.002623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe291f8-2529-42e5-8b4f-3799ced9655c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:16.008987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-dee5d35a-080f-4f8a-aae1-a586ee00b830 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:16.008987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:16.008987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:16.008987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:16.008987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:16.008987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:16.008987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:16.008987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:16.008987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:16.008987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:16.008987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:16.008987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:16.008987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:16.008987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:16.008987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:16.008987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:16.010027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:16.010027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:16.010027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:16.010027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:16.010027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:16.010027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:16.010027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:16.010027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:16.010027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:16.010027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:16.010027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:16.010027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:16.010027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:16.010027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:16.010027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:16.010027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:16.010880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:16.010880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:16.010880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:16.010880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:16.010880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:16.010880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:16.010880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:16.010880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:16.010880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:16.010880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:16.010880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:16.010880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:16.010880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 338/646] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:16.013268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5d35a-080f-4f8a-aae1-a586ee00b830 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:16.024717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-abe291f8-2529-42e5-8b4f-3799ced9655c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:16.024717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:16.024717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:16.024717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:16.024717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:16.024717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:16.024717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:16.024717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:16.024717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:16.024717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:16.024717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:16.024717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:16.024717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:16.024717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:16.024717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:16.025740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:16.025740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:16.025740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:16.025740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:16.025740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:16.025740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:16.025740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:16.025740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:16.025740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:16.025740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:16.025740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:16.025740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:16.025740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:16.025740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:16.025740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:16.025740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:16.025740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:16.026774 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:16.026774 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:16.026774 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:16.026774 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:16.026774 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 309/647] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:16.027200 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe291f8-2529-42e5-8b4f-3799ced9655c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:16.054802 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:16.116567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:16.147057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5d35a-080f-4f8a-aae1-a586ee00b830 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:16.150411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe291f8-2529-42e5-8b4f-3799ced9655c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:16.187401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba179297-1320-4fa7-acdc-d2c42cfa0970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:16.187660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb00e1-5943-4133-843d-f3693d090996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:16.188076 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba179297-1320-4fa7-acdc-d2c42cfa0970 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:16.188281 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb00e1-5943-4133-843d-f3693d090996 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:16.188382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba179297-1320-4fa7-acdc-d2c42cfa0970 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:16.188562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb00e1-5943-4133-843d-f3693d090996 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:16.188671 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba179297-1320-4fa7-acdc-d2c42cfa0970 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:16.188889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb00e1-5943-4133-843d-f3693d090996 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:16.189460 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba179297-1320-4fa7-acdc-d2c42cfa0970 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:16.189678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb00e1-5943-4133-843d-f3693d090996 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:16.189773 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba179297-1320-4fa7-acdc-d2c42cfa0970 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:16.190001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb00e1-5943-4133-843d-f3693d090996 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:16.191280 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba179297-1320-4fa7-acdc-d2c42cfa0970 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:16.191458 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb00e1-5943-4133-843d-f3693d090996 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:16.210420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 310/648] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:16.210420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb00e1-5943-4133-843d-f3693d090996 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:16.215578 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 339/649] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Jul 12 13:29:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:16.220254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba179297-1320-4fa7-acdc-d2c42cfa0970 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:16.298402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:16.320163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb00e1-5943-4133-843d-f3693d090996 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:16.355856 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba179297-1320-4fa7-acdc-d2c42cfa0970 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:16.356351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145fc55-0ea5-46f4-aec0-f041a9c4c9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:16.357029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145fc55-0ea5-46f4-aec0-f041a9c4c9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:16.357306 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145fc55-0ea5-46f4-aec0-f041a9c4c9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:16.357587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145fc55-0ea5-46f4-aec0-f041a9c4c9d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:16.358381 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145fc55-0ea5-46f4-aec0-f041a9c4c9d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:16.358822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145fc55-0ea5-46f4-aec0-f041a9c4c9d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:16.360079 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145fc55-0ea5-46f4-aec0-f041a9c4c9d6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:16.368355 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 311/650] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:29:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:16.369171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145fc55-0ea5-46f4-aec0-f041a9c4c9d6 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:16.417811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60ca12-4865-4328-8a64-b91fd7274ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:16.418553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60ca12-4865-4328-8a64-b91fd7274ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:16.419727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60ca12-4865-4328-8a64-b91fd7274ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:16.420019 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60ca12-4865-4328-8a64-b91fd7274ff2 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:16.421021 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60ca12-4865-4328-8a64-b91fd7274ff2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:16.421303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60ca12-4865-4328-8a64-b91fd7274ff2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:16.422734 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60ca12-4865-4328-8a64-b91fd7274ff2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:16.426562 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:16.438956 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 340/651] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:16.444628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60ca12-4865-4328-8a64-b91fd7274ff2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:16.510787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145fc55-0ea5-46f4-aec0-f041a9c4c9d6 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:16.555326 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a7330-033a-4069-a928-bf73e58823f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:16.556137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a7330-033a-4069-a928-bf73e58823f3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:16.556516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a7330-033a-4069-a928-bf73e58823f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:16.556788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a7330-033a-4069-a928-bf73e58823f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:16.557833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a7330-033a-4069-a928-bf73e58823f3 None admin] RBAC: Adding query filter params (role.id=057d9335b3b1412ebfb6bfb27f02f96a, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=0d3915cde0d04bcfb9b5ce8c89af1e5b) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:16.558206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a7330-033a-4069-a928-bf73e58823f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:16.560296 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a7330-033a-4069-a928-bf73e58823f3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:16.567525 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 312/652] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:16 2022] GET /identity/v3/role_assignments?user.id=0d3915cde0d04bcfb9b5ce8c89af1e5b&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=057d9335b3b1412ebfb6bfb27f02f96a => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:16.568981 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a7330-033a-4069-a928-bf73e58823f3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:16.582262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:16.582927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60ca12-4865-4328-8a64-b91fd7274ff2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:16.651999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2e72c-a550-4108-a56c-910c845a78b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:16.654001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2e72c-a550-4108-a56c-910c845a78b0 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:16.654818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2e72c-a550-4108-a56c-910c845a78b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:16.655565 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2e72c-a550-4108-a56c-910c845a78b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:16.742128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a7330-033a-4069-a928-bf73e58823f3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:16.773635 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:16.784114 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2e72c-a550-4108-a56c-910c845a78b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:16.785623 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2e72c-a550-4108-a56c-910c845a78b0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:16.790092 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba5797-6167-469e-b34d-3f67159cd7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:16.790946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba5797-6167-469e-b34d-3f67159cd7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:16.791233 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba5797-6167-469e-b34d-3f67159cd7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:16.791511 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba5797-6167-469e-b34d-3f67159cd7e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:16.807603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba5797-6167-469e-b34d-3f67159cd7e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:16.810312 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba5797-6167-469e-b34d-3f67159cd7e9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:16.834812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-7bba5797-6167-469e-b34d-3f67159cd7e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:16.834812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:16.834812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:16.834812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:16.834812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:16.834812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:16.834812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:16.834812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:16.834812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:16.834812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:16.834812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:16.834812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:16.834812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:16.834812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:16.834812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:16.836321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:16.836321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:16.836321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:16.836321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:16.836321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:16.836321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:16.836321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:16.836321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:16.836321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:16.836321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:16.836321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:16.836321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:16.836321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:16.836321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:16.836321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:16.836321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:16.836321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:16.838592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:16.838592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:16.838592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:16.838592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:16.838592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 313/653] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:16.838592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba5797-6167-469e-b34d-3f67159cd7e9 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:16.933417 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:16.988579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 341/654] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:29:16.990217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2e72c-a550-4108-a56c-910c845a78b0 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:17.006440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba5797-6167-469e-b34d-3f67159cd7e9 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:17.045929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52dc788-44b8-4a37-9627-727269766a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:17.046828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dc788-44b8-4a37-9627-727269766a3b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.047120 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dc788-44b8-4a37-9627-727269766a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.047477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dc788-44b8-4a37-9627-727269766a3b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.056257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52dc788-44b8-4a37-9627-727269766a3b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:17.057816 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52dc788-44b8-4a37-9627-727269766a3b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:17.073117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-f52dc788-44b8-4a37-9627-727269766a3b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 13:29:17.073117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:17.073117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:17.073117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:17.073117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:17.073117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:17.073117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:17.073117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:17.073117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:17.073117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:17.073117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:17.073117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:17.073117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:17.073117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:17.073117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:17.073117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:17.075491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:17.077426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:17.077426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:17.077426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:17.077426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:17.077426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:17.077426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:17.077426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:17.077426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:17.077426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 13:29:17.077426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:17.077426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 314/655] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:29:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:17.077426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52dc788-44b8-4a37-9627-727269766a3b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:17.107012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:17.114324 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2e72c-a550-4108-a56c-910c845a78b0 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:17.153156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457f145-9b26-4659-abec-558abe5f7c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:17.154278 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457f145-9b26-4659-abec-558abe5f7c28 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.154278 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457f145-9b26-4659-abec-558abe5f7c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.154574 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457f145-9b26-4659-abec-558abe5f7c28 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.156667 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457f145-9b26-4659-abec-558abe5f7c28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:17.156667 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457f145-9b26-4659-abec-558abe5f7c28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:17.157033 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457f145-9b26-4659-abec-558abe5f7c28 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:17.171019 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 342/656] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:17.171919 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457f145-9b26-4659-abec-558abe5f7c28 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:17.268851 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52dc788-44b8-4a37-9627-727269766a3b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:17.289273 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457f145-9b26-4659-abec-558abe5f7c28 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:17.304275 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:17.327415 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d5497-0709-4b3f-b19e-161a1ac6e1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:17.328012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d5497-0709-4b3f-b19e-161a1ac6e1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.328294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d5497-0709-4b3f-b19e-161a1ac6e1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.328568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d5497-0709-4b3f-b19e-161a1ac6e1a7 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.329476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d5497-0709-4b3f-b19e-161a1ac6e1a7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:17.329794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d5497-0709-4b3f-b19e-161a1ac6e1a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:17.331382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d5497-0709-4b3f-b19e-161a1ac6e1a7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:17.338957 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694dc64-cd9b-497e-bb64-b726baa9730d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:17.341118 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694dc64-cd9b-497e-bb64-b726baa9730d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.341691 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694dc64-cd9b-497e-bb64-b726baa9730d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.342313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694dc64-cd9b-497e-bb64-b726baa9730d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.355643 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 343/657] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Jul 12 13:29:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:17.356622 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d5497-0709-4b3f-b19e-161a1ac6e1a7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:17.363334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694dc64-cd9b-497e-bb64-b726baa9730d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:17.366080 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694dc64-cd9b-497e-bb64-b726baa9730d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:17.373298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:17.401625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-a694dc64-cd9b-497e-bb64-b726baa9730d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:17.401625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:17.401625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:17.401625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:17.401625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:17.401625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:17.401625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:17.401625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:17.401625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:17.401625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:17.401625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:17.401625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:17.401625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:17.401625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:17.401625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:17.404427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:17.404427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:17.404427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:17.404427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:17.404427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:17.404427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:17.404427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:17.404427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:17.404427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:17.404427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:17.404427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:17.404427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:17.404427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:17.404427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:17.404427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:17.404427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:17.404427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:17.406693 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:17.406693 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:17.406693 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:17.406693 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:17.406693 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 315/658] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:17.406693 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694dc64-cd9b-497e-bb64-b726baa9730d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:17.472800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d5497-0709-4b3f-b19e-161a1ac6e1a7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:17.516126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ff6ca-d8a0-4fcd-825c-a17123532f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:17.517269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ff6ca-d8a0-4fcd-825c-a17123532f3a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.517813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ff6ca-d8a0-4fcd-825c-a17123532f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.518122 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ff6ca-d8a0-4fcd-825c-a17123532f3a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.530855 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ff6ca-d8a0-4fcd-825c-a17123532f3a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:17.533899 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ff6ca-d8a0-4fcd-825c-a17123532f3a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:17.555636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-4a3ff6ca-d8a0-4fcd-825c-a17123532f3a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 13:29:17.555636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:17.555636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:17.555636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:17.555636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:17.555636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:17.555636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:17.555636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:17.555636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:17.555636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:17.555636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:17.555636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:17.555636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:17.555636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:17.555636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:17.555636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:17.557993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:17.560408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:17.560408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:17.560408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:17.560408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:17.560408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:17.560408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:17.560408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:17.560408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:17.560408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 13:29:17.560408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:17.560408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 344/659] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:17.560408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ff6ca-d8a0-4fcd-825c-a17123532f3a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:17.578337 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694dc64-cd9b-497e-bb64-b726baa9730d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:17.616834 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4feea-1b33-43c4-bd0d-3580b046fde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:17.617931 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4feea-1b33-43c4-bd0d-3580b046fde0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.618452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4feea-1b33-43c4-bd0d-3580b046fde0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.618961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4feea-1b33-43c4-bd0d-3580b046fde0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.656231 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:17.666006 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:17.686844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ff6ca-d8a0-4fcd-825c-a17123532f3a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:17.702845 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4feea-1b33-43c4-bd0d-3580b046fde0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:17.704759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4feea-1b33-43c4-bd0d-3580b046fde0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:17.724621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5592af-93e5-4998-bc33-0a2bf74b7c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:17.725354 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5592af-93e5-4998-bc33-0a2bf74b7c25 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.725861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5592af-93e5-4998-bc33-0a2bf74b7c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.726181 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5592af-93e5-4998-bc33-0a2bf74b7c25 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.726925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5592af-93e5-4998-bc33-0a2bf74b7c25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:17.727242 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5592af-93e5-4998-bc33-0a2bf74b7c25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:17.728649 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5592af-93e5-4998-bc33-0a2bf74b7c25 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:17.742287 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 345/660] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:17.742522 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5592af-93e5-4998-bc33-0a2bf74b7c25 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:17.818059 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 316/661] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:29:17.820599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61313d5-5e60-4e0d-9141-e932051ad40c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.821142 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61313d5-5e60-4e0d-9141-e932051ad40c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.821786 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61313d5-5e60-4e0d-9141-e932051ad40c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.824215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 317/662] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:17.826217 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61313d5-5e60-4e0d-9141-e932051ad40c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:17.857170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5592af-93e5-4998-bc33-0a2bf74b7c25 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:17.896911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3be05-eed4-4eb2-a616-1d38ba7192b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:17.897699 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3be05-eed4-4eb2-a616-1d38ba7192b2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.898006 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3be05-eed4-4eb2-a616-1d38ba7192b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.898410 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3be05-eed4-4eb2-a616-1d38ba7192b2 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.899246 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3be05-eed4-4eb2-a616-1d38ba7192b2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:17.899560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3be05-eed4-4eb2-a616-1d38ba7192b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:17.901133 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3be05-eed4-4eb2-a616-1d38ba7192b2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:17.903061 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:17.913108 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:17.921874 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 346/663] 10.222.0.52 () {56 vars in 1139 bytes} [Tue Jul 12 13:29:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:17.922738 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3be05-eed4-4eb2-a616-1d38ba7192b2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:17.954077 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61313d5-5e60-4e0d-9141-e932051ad40c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:17.990844 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4a636-1e9a-4c2c-9bde-a242fc7b36be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:17.991738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4a636-1e9a-4c2c-9bde-a242fc7b36be None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.991952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4a636-1e9a-4c2c-9bde-a242fc7b36be None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.992183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4a636-1e9a-4c2c-9bde-a242fc7b36be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:18.000268 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4a636-1e9a-4c2c-9bde-a242fc7b36be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:18.001985 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4a636-1e9a-4c2c-9bde-a242fc7b36be None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:18.017226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-99b4a636-1e9a-4c2c-9bde-a242fc7b36be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:18.017226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:18.017226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:18.017226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:18.017226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:18.017226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:18.017226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:18.017226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:18.017226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:18.017226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:18.017226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:18.017226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:18.017226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:18.017226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:18.017226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:18.018330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:18.018330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:18.018330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:18.018330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:18.018330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:18.018330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:18.018330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:18.018330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:18.018330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:18.018330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:18.018330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:18.018330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:18.018330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:18.018330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:18.018330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:18.018330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:18.018330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:18.019258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:18.019258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:18.019258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:18.019258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:18.019258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:18.019258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:18.019258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:18.019258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:18.019258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 318/664] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:18.019742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fed39c-3c9b-4429-95f2-942edc741cda None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:18.020254 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fed39c-3c9b-4429-95f2-942edc741cda None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:18.020254 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fed39c-3c9b-4429-95f2-942edc741cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:18.038815 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-95fed39c-3c9b-4429-95f2-942edc741cda None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:18.040336 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-95fed39c-3c9b-4429-95f2-942edc741cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:18.041457 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3be05-eed4-4eb2-a616-1d38ba7192b2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:18.083665 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd4242-0caa-4a50-8e3b-d8d967eac137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:18.084492 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd4242-0caa-4a50-8e3b-d8d967eac137 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:18.084773 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd4242-0caa-4a50-8e3b-d8d967eac137 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:18.085045 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd4242-0caa-4a50-8e3b-d8d967eac137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:18.096419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd4242-0caa-4a50-8e3b-d8d967eac137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:18.098044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd4242-0caa-4a50-8e3b-d8d967eac137 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:18.117833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-b1bd4242-0caa-4a50-8e3b-d8d967eac137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:18.117833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:18.117833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:18.117833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:18.117833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:18.117833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:18.117833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:18.117833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:18.117833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:18.117833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:18.117833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:18.117833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:18.117833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:18.117833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:18.117833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:18.118990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:18.118990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:18.118990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:18.118990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:18.118990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:18.118990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:18.118990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:18.118990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:18.118990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:18.118990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:18.118990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:18.118990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:18.118990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:18.118990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:18.118990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:18.118990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:18.118990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:18.120058 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:18.120058 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:18.120058 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:18.120058 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:18.122334 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 347/665] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:18.123606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd4242-0caa-4a50-8e3b-d8d967eac137 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:18.128169 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 319/666] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:18.129017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fed39c-3c9b-4429-95f2-942edc741cda None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:18.180920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:18.197922 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:18.243277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd4242-0caa-4a50-8e3b-d8d967eac137 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:18.245317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fed39c-3c9b-4429-95f2-942edc741cda None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:18.282623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e882a-b116-4aac-bcfc-af6f9c3a5512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:18.283282 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e882a-b116-4aac-bcfc-af6f9c3a5512 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:18.283563 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e882a-b116-4aac-bcfc-af6f9c3a5512 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:18.283833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e882a-b116-4aac-bcfc-af6f9c3a5512 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:18.284622 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e882a-b116-4aac-bcfc-af6f9c3a5512 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:18.284931 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e882a-b116-4aac-bcfc-af6f9c3a5512 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:18.285282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03555e-cbc6-4e7f-aad1-e79a11b1eb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:18.286159 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03555e-cbc6-4e7f-aad1-e79a11b1eb05 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:18.286637 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03555e-cbc6-4e7f-aad1-e79a11b1eb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:18.287240 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e882a-b116-4aac-bcfc-af6f9c3a5512 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:18.287475 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03555e-cbc6-4e7f-aad1-e79a11b1eb05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:18.300198 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 320/667] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:18.301030 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e882a-b116-4aac-bcfc-af6f9c3a5512 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:18.377172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03555e-cbc6-4e7f-aad1-e79a11b1eb05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:18.378555 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03555e-cbc6-4e7f-aad1-e79a11b1eb05 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:18.425305 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e882a-b116-4aac-bcfc-af6f9c3a5512 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:18.432147 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:18.454486 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:18.474824 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408feb43-12fc-4852-8f44-4bd61163ed57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:18.475512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408feb43-12fc-4852-8f44-4bd61163ed57 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:18.475797 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408feb43-12fc-4852-8f44-4bd61163ed57 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:18.476072 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408feb43-12fc-4852-8f44-4bd61163ed57 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:18.476818 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408feb43-12fc-4852-8f44-4bd61163ed57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:18.477127 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408feb43-12fc-4852-8f44-4bd61163ed57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:18.479153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408feb43-12fc-4852-8f44-4bd61163ed57 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:18.495120 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 321/668] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:18.496085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408feb43-12fc-4852-8f44-4bd61163ed57 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:18.532607 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 348/669] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:29:18.532607 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03555e-cbc6-4e7f-aad1-e79a11b1eb05 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:18.636167 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408feb43-12fc-4852-8f44-4bd61163ed57 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:18.671869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03555e-cbc6-4e7f-aad1-e79a11b1eb05 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:18.686055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48f9bc-8d55-4fbd-97cb-56834e2a70d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:18.686888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48f9bc-8d55-4fbd-97cb-56834e2a70d0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:18.687175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48f9bc-8d55-4fbd-97cb-56834e2a70d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:18.687450 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48f9bc-8d55-4fbd-97cb-56834e2a70d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:18.700383 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48f9bc-8d55-4fbd-97cb-56834e2a70d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:18.709082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-cd48f9bc-8d55-4fbd-97cb-56834e2a70d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:18.709082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:18.709082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:18.709082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:18.709082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:18.709082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:18.709082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:18.709082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:18.709082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:18.709082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:18.709082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:18.709082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:18.709082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:18.709082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:18.709082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:18.709082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:18.710279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:18.710279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:18.710279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:18.710279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:18.710279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:18.710279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:18.710279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:18.710279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:18.710279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:18.710279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:18.710279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:18.710279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:18.710279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:18.710279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:18.710279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:18.710279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:18.712631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:18.712631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:18.712631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:18.712631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:18.712631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:18.712631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:18.712631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:18.712631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:18.712631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:18.712631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:18.712631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:18.712631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:18.712631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 322/670] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:18.712631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48f9bc-8d55-4fbd-97cb-56834e2a70d0 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:18.726439 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9de06-1fcf-4bce-a274-09b6f67e81e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:18.727152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9de06-1fcf-4bce-a274-09b6f67e81e8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:18.727444 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9de06-1fcf-4bce-a274-09b6f67e81e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:18.727756 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9de06-1fcf-4bce-a274-09b6f67e81e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:18.728672 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9de06-1fcf-4bce-a274-09b6f67e81e8 None admin] RBAC: Adding query filter params (role.id=057d9335b3b1412ebfb6bfb27f02f96a, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=ebb4da7b00c64fafbb3724bae25b2853) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:18.728992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9de06-1fcf-4bce-a274-09b6f67e81e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:18.730545 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9de06-1fcf-4bce-a274-09b6f67e81e8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:18.746308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 349/671] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:18 2022] GET /identity/v3/role_assignments?user.id=ebb4da7b00c64fafbb3724bae25b2853&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=057d9335b3b1412ebfb6bfb27f02f96a => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:18.747191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9de06-1fcf-4bce-a274-09b6f67e81e8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:18.767689 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:18.783972 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:18.888049 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48f9bc-8d55-4fbd-97cb-56834e2a70d0 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:18.906463 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9de06-1fcf-4bce-a274-09b6f67e81e8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:18.927580 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e23f9-dad9-4ace-b408-c226e166eae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:18.928405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e23f9-dad9-4ace-b408-c226e166eae4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:18.929433 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e23f9-dad9-4ace-b408-c226e166eae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:18.929705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e23f9-dad9-4ace-b408-c226e166eae4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:18.942828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e23f9-dad9-4ace-b408-c226e166eae4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:18.945288 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e23f9-dad9-4ace-b408-c226e166eae4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:18.950946 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea7f94-9017-4197-93cf-cf48c8de2b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:18.951764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea7f94-9017-4197-93cf-cf48c8de2b1d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:18.952049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea7f94-9017-4197-93cf-cf48c8de2b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:18.952325 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea7f94-9017-4197-93cf-cf48c8de2b1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:18.962351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-173e23f9-dad9-4ace-b408-c226e166eae4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:18.962351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:18.962351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:18.962351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:18.962351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:18.962351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:18.962351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:18.962351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:18.962351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:18.962351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:18.962351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:18.962351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:18.962351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:18.962351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:18.962351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:18.963685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:18.963685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:18.963685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:18.963685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:18.963685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:18.963685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:18.963685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:18.963685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:18.963685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:18.963685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:18.963685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:18.963685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:18.963685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:18.963685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:18.963685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:18.963685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:18.963685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:18.964648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:18.964648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:18.964648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:18.964648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:18.964648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:18.964648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:18.964648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:18.964648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:18.964648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 323/672] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:18.966207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e23f9-dad9-4ace-b408-c226e166eae4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:18.970838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea7f94-9017-4197-93cf-cf48c8de2b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:18.972453 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea7f94-9017-4197-93cf-cf48c8de2b1d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:18.994425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-b6ea7f94-9017-4197-93cf-cf48c8de2b1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:18.994425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:18.994425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:18.994425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:18.994425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:18.994425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:18.994425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:18.994425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:18.994425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:18.994425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:18.994425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:18.994425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:18.994425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:18.994425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:18.994425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:18.995398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:18.995398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:18.995398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:18.995398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:18.995398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:18.995398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:18.995398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:18.995398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:18.995398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:18.995398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:18.995398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:18.995398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:18.995398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:18.995398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:18.995398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:18.995398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:18.995398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:18.996364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:18.996364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:18.996364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:18.996364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:18.996364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 350/673] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:18.996641 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea7f94-9017-4197-93cf-cf48c8de2b1d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:19.098954 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:19.103279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e23f9-dad9-4ace-b408-c226e166eae4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:19.111026 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:19.164869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea7f94-9017-4197-93cf-cf48c8de2b1d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:19.165934 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91bdfd-1fd4-4c4d-ac8d-bffad22aebfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:19.166826 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91bdfd-1fd4-4c4d-ac8d-bffad22aebfd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.167119 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91bdfd-1fd4-4c4d-ac8d-bffad22aebfd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.168340 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91bdfd-1fd4-4c4d-ac8d-bffad22aebfd None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.169123 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91bdfd-1fd4-4c4d-ac8d-bffad22aebfd None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:19.169433 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91bdfd-1fd4-4c4d-ac8d-bffad22aebfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:19.170860 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91bdfd-1fd4-4c4d-ac8d-bffad22aebfd None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:19.186239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 324/674] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:29:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:19.186239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91bdfd-1fd4-4c4d-ac8d-bffad22aebfd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:19.212747 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe376167-e9c6-442a-bec1-39e09290ff6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:19.213428 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe376167-e9c6-442a-bec1-39e09290ff6e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.213709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe376167-e9c6-442a-bec1-39e09290ff6e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.214011 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe376167-e9c6-442a-bec1-39e09290ff6e None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.214829 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe376167-e9c6-442a-bec1-39e09290ff6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:19.215189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe376167-e9c6-442a-bec1-39e09290ff6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:19.216684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe376167-e9c6-442a-bec1-39e09290ff6e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:19.230665 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 351/675] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:19.232092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc0088-ea5d-45a0-a958-558e7308cc16 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.232330 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc0088-ea5d-45a0-a958-558e7308cc16 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.232619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc0088-ea5d-45a0-a958-558e7308cc16 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.240060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 352/676] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:19.240060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc0088-ea5d-45a0-a958-558e7308cc16 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:19.317144 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91bdfd-1fd4-4c4d-ac8d-bffad22aebfd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:19.367358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91280c55-cbad-494a-bbe5-837f614c2a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:19.368022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91280c55-cbad-494a-bbe5-837f614c2a13 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.368303 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91280c55-cbad-494a-bbe5-837f614c2a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.368574 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91280c55-cbad-494a-bbe5-837f614c2a13 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.369302 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91280c55-cbad-494a-bbe5-837f614c2a13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:19.369604 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91280c55-cbad-494a-bbe5-837f614c2a13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:19.371020 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91280c55-cbad-494a-bbe5-837f614c2a13 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:19.375141 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc0088-ea5d-45a0-a958-558e7308cc16 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:19.384564 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 325/677] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:19.385435 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91280c55-cbad-494a-bbe5-837f614c2a13 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:19.403987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:19.424505 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:19.424889 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78404181-ce2e-4073-893a-18e7db066156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:19.425820 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78404181-ce2e-4073-893a-18e7db066156 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.426480 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78404181-ce2e-4073-893a-18e7db066156 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.426480 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78404181-ce2e-4073-893a-18e7db066156 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.440929 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78404181-ce2e-4073-893a-18e7db066156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:19.442467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78404181-ce2e-4073-893a-18e7db066156 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:19.461701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-78404181-ce2e-4073-893a-18e7db066156 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:19.461701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:19.461701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:19.461701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:19.461701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:19.461701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:19.461701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:19.461701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:19.461701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:19.461701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:19.461701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:19.461701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:19.461701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:19.461701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:19.461701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:19.462999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:19.462999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:19.462999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:19.462999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:19.462999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:19.462999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:19.462999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:19.462999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:19.462999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:19.462999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:19.462999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:19.462999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:19.462999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:19.462999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:19.462999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:19.462999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:19.462999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:19.464106 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:19.464106 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:19.464106 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:19.464106 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:19.464106 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 353/678] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:19.464395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a27a-b60c-45e5-941d-66f3d4341053 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.464472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a27a-b60c-45e5-941d-66f3d4341053 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.465225 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a27a-b60c-45e5-941d-66f3d4341053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.483128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-da45a27a-b60c-45e5-941d-66f3d4341053 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:19.484355 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-da45a27a-b60c-45e5-941d-66f3d4341053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:19.535467 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91280c55-cbad-494a-bbe5-837f614c2a13 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:19.562076 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 354/679] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:19.563506 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45a27a-b60c-45e5-941d-66f3d4341053 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:19.599956 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747a633-66e4-4527-b725-6e92342e5fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:19.601383 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747a633-66e4-4527-b725-6e92342e5fba None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.601892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747a633-66e4-4527-b725-6e92342e5fba None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.602680 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747a633-66e4-4527-b725-6e92342e5fba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.604197 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747a633-66e4-4527-b725-6e92342e5fba None admin] RBAC: Adding query filter params (role.id=057d9335b3b1412ebfb6bfb27f02f96a, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=35336384bbca475082be349df406ae3f) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:19.604782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747a633-66e4-4527-b725-6e92342e5fba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:19.614328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747a633-66e4-4527-b725-6e92342e5fba None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:19.622095 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 326/680] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:19 2022] GET /identity/v3/role_assignments?user.id=35336384bbca475082be349df406ae3f&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=057d9335b3b1412ebfb6bfb27f02f96a => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:19.622942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747a633-66e4-4527-b725-6e92342e5fba None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:19.749142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45a27a-b60c-45e5-941d-66f3d4341053 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:19.755605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:19.789873 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:19.805659 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede52f84-dc28-4aa3-97c9-20272ac2b502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:19.806811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede52f84-dc28-4aa3-97c9-20272ac2b502 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.807243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede52f84-dc28-4aa3-97c9-20272ac2b502 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.807684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede52f84-dc28-4aa3-97c9-20272ac2b502 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.813316 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747a633-66e4-4527-b725-6e92342e5fba None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:19.816969 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede52f84-dc28-4aa3-97c9-20272ac2b502 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:19.818679 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede52f84-dc28-4aa3-97c9-20272ac2b502 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:19.836442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-ede52f84-dc28-4aa3-97c9-20272ac2b502 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:19.836442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:19.836442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:19.836442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:19.836442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:19.836442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:19.836442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:19.836442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:19.836442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:19.836442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:19.836442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:19.836442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:19.836442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:19.836442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:19.836442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:19.839083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:19.839083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:19.839083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:19.839083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:19.839083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:19.839083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:19.839083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:19.839083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:19.839083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:19.839083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:19.839083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:19.839083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:19.839083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:19.839083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:19.839083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:19.839083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:19.839083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:19.840633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:19.840633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:19.840633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:19.840633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:19.840633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:19.840633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:19.840633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:19.840633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:19.840633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 355/681] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:19.840633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede52f84-dc28-4aa3-97c9-20272ac2b502 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:19.878413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8326ca-bc0b-4510-b821-7c21d0f8f1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:19.879620 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8326ca-bc0b-4510-b821-7c21d0f8f1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.880078 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8326ca-bc0b-4510-b821-7c21d0f8f1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.880642 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8326ca-bc0b-4510-b821-7c21d0f8f1a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.882001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8326ca-bc0b-4510-b821-7c21d0f8f1a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:19.882515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8326ca-bc0b-4510-b821-7c21d0f8f1a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:19.884074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8326ca-bc0b-4510-b821-7c21d0f8f1a1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:19.899180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 327/682] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:19.905359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8326ca-bc0b-4510-b821-7c21d0f8f1a1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:19.978723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede52f84-dc28-4aa3-97c9-20272ac2b502 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:20.018878 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81453474-66d2-4223-9b47-7020937a3b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:20.019525 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81453474-66d2-4223-9b47-7020937a3b5b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:20.019809 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81453474-66d2-4223-9b47-7020937a3b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:20.020810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81453474-66d2-4223-9b47-7020937a3b5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:20.033822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8326ca-bc0b-4510-b821-7c21d0f8f1a1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:20.034922 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81453474-66d2-4223-9b47-7020937a3b5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:20.043923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-81453474-66d2-4223-9b47-7020937a3b5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:29:20.043923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:20.043923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:20.043923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:20.043923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:20.043923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:20.043923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:20.043923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:20.043923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:20.043923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:20.043923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:20.043923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:20.043923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:20.043923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:20.043923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:20.043923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:20.044986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:20.044986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:20.044986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:20.044986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:20.044986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:20.044986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:20.044986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:20.044986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:20.044986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:20.044986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:20.044986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:20.044986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:20.044986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:20.044986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:20.044986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:20.044986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:20.045843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:20.045843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:20.045843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:20.045843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:20.045843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:20.045843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:20.045843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:20.045843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:20.045843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:20.045843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:20.045843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:29:20.045843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:20.050104 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 356/683] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:29:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:20.051173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81453474-66d2-4223-9b47-7020937a3b5b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:20.075642 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a8ac5-3fa2-4981-b0c4-85233f60f3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:20.076693 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a8ac5-3fa2-4981-b0c4-85233f60f3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:20.077216 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a8ac5-3fa2-4981-b0c4-85233f60f3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:20.077699 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a8ac5-3fa2-4981-b0c4-85233f60f3d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:20.078748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a8ac5-3fa2-4981-b0c4-85233f60f3d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:20.079271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a8ac5-3fa2-4981-b0c4-85233f60f3d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:20.080956 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a8ac5-3fa2-4981-b0c4-85233f60f3d3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:20.086388 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:20.096006 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 328/684] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:20.097214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a8ac5-3fa2-4981-b0c4-85233f60f3d3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:20.120561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:20.206826 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81453474-66d2-4223-9b47-7020937a3b5b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:20.257687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c28c2-9019-48fe-b782-315129e2abfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:20.259331 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c28c2-9019-48fe-b782-315129e2abfa None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:20.259860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c28c2-9019-48fe-b782-315129e2abfa None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:20.260364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c28c2-9019-48fe-b782-315129e2abfa None admin] PATH_INFO: `/v3/users/swift` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:20.270198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c28c2-9019-48fe-b782-315129e2abfa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:20.272124 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c28c2-9019-48fe-b782-315129e2abfa None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:20.276248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a8ac5-3fa2-4981-b0c4-85233f60f3d3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:20.289421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-680c28c2-9019-48fe-b782-315129e2abfa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:29:20.289421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:20.289421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:20.289421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:20.289421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:20.289421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:20.289421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:20.289421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:20.289421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:20.289421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:20.289421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:20.289421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:20.289421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:20.289421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:20.289421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:20.289421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:20.290496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:20.291443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:20.291443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:20.291443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:20.291443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:20.291443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:20.291443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:20.291443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:20.291443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:20.291443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:29:20.291443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:20.298632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 357/685] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:29:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:20.299849 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c28c2-9019-48fe-b782-315129e2abfa None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:20.333525 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1068b-3b39-4511-97cc-627301878e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:20.335061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1068b-3b39-4511-97cc-627301878e16 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:20.335576 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1068b-3b39-4511-97cc-627301878e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:20.336138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1068b-3b39-4511-97cc-627301878e16 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:20.339046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1068b-3b39-4511-97cc-627301878e16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:20.339694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1068b-3b39-4511-97cc-627301878e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:20.345100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1068b-3b39-4511-97cc-627301878e16 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:20.360438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 329/686] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:29:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:20.366996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1068b-3b39-4511-97cc-627301878e16 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:20.428213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:20.464399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c28c2-9019-48fe-b782-315129e2abfa None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:20.491448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:20.521032 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40d954-24bc-42e4-ac3f-522aa2e3d58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:20.521733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40d954-24bc-42e4-ac3f-522aa2e3d58a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:20.522036 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40d954-24bc-42e4-ac3f-522aa2e3d58a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:20.522311 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40d954-24bc-42e4-ac3f-522aa2e3d58a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:20.523159 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40d954-24bc-42e4-ac3f-522aa2e3d58a None admin] RBAC: Adding query filter params (role.id=057d9335b3b1412ebfb6bfb27f02f96a, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=3e833722fe4e493e9ddc4b275632536f) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:20.523465 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40d954-24bc-42e4-ac3f-522aa2e3d58a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:20.524930 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40d954-24bc-42e4-ac3f-522aa2e3d58a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:20.534574 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 358/687] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:20 2022] GET /identity/v3/role_assignments?user.id=3e833722fe4e493e9ddc4b275632536f&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=057d9335b3b1412ebfb6bfb27f02f96a => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:20.535584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40d954-24bc-42e4-ac3f-522aa2e3d58a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:20.554395 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1068b-3b39-4511-97cc-627301878e16 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:20.631186 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adc7d0-6eff-4eae-a171-60548107bc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:20.633240 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adc7d0-6eff-4eae-a171-60548107bc51 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:20.634511 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adc7d0-6eff-4eae-a171-60548107bc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:20.637911 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adc7d0-6eff-4eae-a171-60548107bc51 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:20.642714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92adc7d0-6eff-4eae-a171-60548107bc51 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:20.643703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92adc7d0-6eff-4eae-a171-60548107bc51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:20.646926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92adc7d0-6eff-4eae-a171-60548107bc51 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:20.717173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 330/688] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Jul 12 13:29:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:20.720243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adc7d0-6eff-4eae-a171-60548107bc51 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:20.767480 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40d954-24bc-42e4-ac3f-522aa2e3d58a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:20.810613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b619d21-7687-4303-8d47-9401e60f6448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:20.812096 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b619d21-7687-4303-8d47-9401e60f6448 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:20.812609 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b619d21-7687-4303-8d47-9401e60f6448 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:20.813114 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b619d21-7687-4303-8d47-9401e60f6448 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:20.837314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b619d21-7687-4303-8d47-9401e60f6448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:20.839069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b619d21-7687-4303-8d47-9401e60f6448 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:20.848731 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:20.864255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-7b619d21-7687-4303-8d47-9401e60f6448 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:20.864255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:20.864255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:20.864255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:20.864255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:20.864255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:20.864255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:20.864255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:20.864255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:20.864255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:20.864255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:20.864255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:20.864255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:20.864255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:20.864255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:20.865224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:20.865224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:20.865224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:20.865224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:20.865224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:20.865224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:20.865224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:20.865224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:20.865224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:20.865224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:20.865224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:20.865224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:20.865224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:20.865224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:20.865224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:20.865224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:20.865224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:20.866177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:20.866177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:20.866177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:20.866177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:20.871750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 359/689] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:20.873232 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b619d21-7687-4303-8d47-9401e60f6448 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:20.890003 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adc7d0-6eff-4eae-a171-60548107bc51 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:20.955583 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:20.982415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42c017-0c53-4611-8758-8139b1428c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:20.983524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42c017-0c53-4611-8758-8139b1428c9a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:20.984041 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42c017-0c53-4611-8758-8139b1428c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:20.984703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42c017-0c53-4611-8758-8139b1428c9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:20.998419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b619d21-7687-4303-8d47-9401e60f6448 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:21.056949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e1429-3979-4bde-872e-f14df2d27eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:21.057629 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e1429-3979-4bde-872e-f14df2d27eab None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.057914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e1429-3979-4bde-872e-f14df2d27eab None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.058221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e1429-3979-4bde-872e-f14df2d27eab None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.059526 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e1429-3979-4bde-872e-f14df2d27eab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:21.059838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e1429-3979-4bde-872e-f14df2d27eab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.061338 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e1429-3979-4bde-872e-f14df2d27eab None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:21.079341 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 360/690] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:21.080823 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aadc80-cd39-4f4f-9dcf-d34a3240003b None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.081118 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aadc80-cd39-4f4f-9dcf-d34a3240003b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.081399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aadc80-cd39-4f4f-9dcf-d34a3240003b None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.083102 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 361/691] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:21.083835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aadc80-cd39-4f4f-9dcf-d34a3240003b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:21.088351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42c017-0c53-4611-8758-8139b1428c9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.089661 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42c017-0c53-4611-8758-8139b1428c9a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:21.170425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:21.216414 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aadc80-cd39-4f4f-9dcf-d34a3240003b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:21.229270 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 331/692] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:29:21.231604 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e22fe9-2f90-448a-bb52-479be86d1277 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.231892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e22fe9-2f90-448a-bb52-479be86d1277 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.232332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e22fe9-2f90-448a-bb52-479be86d1277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.253093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d5e22fe9-2f90-448a-bb52-479be86d1277 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:21.254630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d5e22fe9-2f90-448a-bb52-479be86d1277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:21.254744 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c05d6-990e-4535-bf06-e3138303d1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:21.255574 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c05d6-990e-4535-bf06-e3138303d1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.255849 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c05d6-990e-4535-bf06-e3138303d1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.256175 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c05d6-990e-4535-bf06-e3138303d1a5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.264443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c05d6-990e-4535-bf06-e3138303d1a5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.265976 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c05d6-990e-4535-bf06-e3138303d1a5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:21.278743 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:21.282025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-b72c05d6-990e-4535-bf06-e3138303d1a5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:29:21.282025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:21.282025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:21.282025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:21.282025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:21.282025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:21.282025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:21.282025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:21.282025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:21.282025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:21.282025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:21.282025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:21.282025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:21.282025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:21.282025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:21.282025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:21.284233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:21.286567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:21.286567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:21.286567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:21.286567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:21.286567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:21.286567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:21.286567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:21.286567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:21.286567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:29:21.286567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:21.287928 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 362/693] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jul 12 13:29:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:21.298012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c05d6-990e-4535-bf06-e3138303d1a5 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:21.334804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 332/694] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:21.335994 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e22fe9-2f90-448a-bb52-479be86d1277 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:21.424520 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c05d6-990e-4535-bf06-e3138303d1a5 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:21.445545 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:21.451760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e22fe9-2f90-448a-bb52-479be86d1277 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:21.465712 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e85bab-6adb-434b-87be-2a96d43838bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:21.466563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e85bab-6adb-434b-87be-2a96d43838bb None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.466832 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e85bab-6adb-434b-87be-2a96d43838bb None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.467102 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e85bab-6adb-434b-87be-2a96d43838bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.478382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e85bab-6adb-434b-87be-2a96d43838bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.479925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e85bab-6adb-434b-87be-2a96d43838bb None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:21.493707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9de9e-74c0-454e-bab3-91e038d34626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:21.494792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9de9e-74c0-454e-bab3-91e038d34626 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.495284 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9de9e-74c0-454e-bab3-91e038d34626 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.495764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9de9e-74c0-454e-bab3-91e038d34626 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.497332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff9de9e-74c0-454e-bab3-91e038d34626 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:21.497852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff9de9e-74c0-454e-bab3-91e038d34626 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.499565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff9de9e-74c0-454e-bab3-91e038d34626 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:21.500811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-95e85bab-6adb-434b-87be-2a96d43838bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:21.500811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:21.500811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:21.500811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:21.500811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:21.500811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:21.500811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:21.500811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:21.500811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:21.500811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:21.500811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:21.500811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:21.500811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:21.500811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:21.500811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:21.501934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:21.501934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:21.501934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:21.501934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:21.501934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:21.501934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:21.501934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:21.501934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:21.501934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:21.501934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:21.501934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:21.501934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:21.501934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:21.501934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:21.501934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:21.501934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:21.501934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:21.502872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:21.502872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:21.502872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:21.502872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:21.507041 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 363/695] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:21.507971 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e85bab-6adb-434b-87be-2a96d43838bb None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:21.523818 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 333/696] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jul 12 13:29:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:21.525027 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9de9e-74c0-454e-bab3-91e038d34626 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:21.543043 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:21.627883 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e85bab-6adb-434b-87be-2a96d43838bb None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:21.644155 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9de9e-74c0-454e-bab3-91e038d34626 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:21.664248 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037eface-0402-4f89-8139-4a7f85d3d2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:21.664979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037eface-0402-4f89-8139-4a7f85d3d2da None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.665265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037eface-0402-4f89-8139-4a7f85d3d2da None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.665542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037eface-0402-4f89-8139-4a7f85d3d2da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.677746 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037eface-0402-4f89-8139-4a7f85d3d2da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.679423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037eface-0402-4f89-8139-4a7f85d3d2da None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:21.686444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc9453-302b-4b9c-b5bd-af438b1ca8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:21.687548 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc9453-302b-4b9c-b5bd-af438b1ca8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.688094 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc9453-302b-4b9c-b5bd-af438b1ca8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.688626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc9453-302b-4b9c-b5bd-af438b1ca8f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.689633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc9453-302b-4b9c-b5bd-af438b1ca8f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:21.690173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc9453-302b-4b9c-b5bd-af438b1ca8f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.691796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc9453-302b-4b9c-b5bd-af438b1ca8f4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:21.698880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-037eface-0402-4f89-8139-4a7f85d3d2da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:21.698880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:21.698880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:21.698880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:21.698880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:21.698880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:21.698880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:21.698880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:21.698880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:21.698880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:21.698880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:21.698880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:21.698880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:21.698880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:21.698880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:21.700367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:21.700367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:21.700367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:21.700367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:21.700367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:21.700367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:21.700367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:21.700367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:21.700367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:21.700367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:21.700367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:21.700367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:21.700367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:21.700367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:21.700367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:21.700367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:21.700367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:21.701282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:21.701282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:21.701282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:21.701282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:21.705588 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 364/697] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:21.706554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037eface-0402-4f89-8139-4a7f85d3d2da None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:21.709175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 334/698] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:21.710523 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc9453-302b-4b9c-b5bd-af438b1ca8f4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:21.739028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:21.799039 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:21.820902 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037eface-0402-4f89-8139-4a7f85d3d2da None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:21.829123 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc9453-302b-4b9c-b5bd-af438b1ca8f4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:21.858323 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10649b2a-6ef1-45c6-91ad-3da5033cdb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:21.858977 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10649b2a-6ef1-45c6-91ad-3da5033cdb1c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.859262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10649b2a-6ef1-45c6-91ad-3da5033cdb1c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.859606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10649b2a-6ef1-45c6-91ad-3da5033cdb1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.872534 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5342fc-b447-4677-811c-8745c347a1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:21.873659 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5342fc-b447-4677-811c-8745c347a1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.874242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5342fc-b447-4677-811c-8745c347a1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.874725 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5342fc-b447-4677-811c-8745c347a1c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.875833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5342fc-b447-4677-811c-8745c347a1c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:21.876452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5342fc-b447-4677-811c-8745c347a1c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.878449 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5342fc-b447-4677-811c-8745c347a1c4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:21.894008 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 335/699] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:21.895416 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5342fc-b447-4677-811c-8745c347a1c4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:21.944312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10649b2a-6ef1-45c6-91ad-3da5033cdb1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.945922 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10649b2a-6ef1-45c6-91ad-3da5033cdb1c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:21.975256 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:22.013117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5342fc-b447-4677-811c-8745c347a1c4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:22.053122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83ea30-5ffe-4d31-b113-754fcb22d305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:22.054518 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83ea30-5ffe-4d31-b113-754fcb22d305 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:22.055348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83ea30-5ffe-4d31-b113-754fcb22d305 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:22.055813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83ea30-5ffe-4d31-b113-754fcb22d305 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:22.059149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:22.066942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83ea30-5ffe-4d31-b113-754fcb22d305 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:22.068668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83ea30-5ffe-4d31-b113-754fcb22d305 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:22.068847 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 365/700] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:29:22.069773 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10649b2a-6ef1-45c6-91ad-3da5033cdb1c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:22.089193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-8f83ea30-5ffe-4d31-b113-754fcb22d305 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:22.089193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:22.089193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:22.089193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:22.089193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:22.089193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:22.089193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:22.089193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:22.089193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:22.089193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:22.089193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:22.089193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:22.089193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:22.089193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:22.089193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:22.091475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:22.091475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:22.091475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:22.091475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:22.091475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:22.091475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:22.091475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:22.091475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:22.091475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:22.091475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:22.091475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:22.091475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:22.091475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:22.091475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:22.091475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:22.091475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:22.091475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:22.092633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:22.092633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:22.092633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:22.092633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:22.092633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:22.092633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:22.092633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:22.092633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:22.095381 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 336/701] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:22.097054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83ea30-5ffe-4d31-b113-754fcb22d305 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:22.201573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10649b2a-6ef1-45c6-91ad-3da5033cdb1c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:22.221325 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83ea30-5ffe-4d31-b113-754fcb22d305 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:22.228812 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:22.241566 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3ab26-1c0c-4a83-96e9-cd752ad3580b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:22.242285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3ab26-1c0c-4a83-96e9-cd752ad3580b None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:22.242573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3ab26-1c0c-4a83-96e9-cd752ad3580b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:22.242850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3ab26-1c0c-4a83-96e9-cd752ad3580b None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:22.243555 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3ab26-1c0c-4a83-96e9-cd752ad3580b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:22.245021 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3ab26-1c0c-4a83-96e9-cd752ad3580b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:22.267507 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0e63c-1dc2-4333-9586-3f3ed7917c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:22.268323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0e63c-1dc2-4333-9586-3f3ed7917c93 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:22.268606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0e63c-1dc2-4333-9586-3f3ed7917c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:22.268883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0e63c-1dc2-4333-9586-3f3ed7917c93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:22.282293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 366/702] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:29:22 2022] POST /identity/v3/users => generated 254 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:22.282293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3ab26-1c0c-4a83-96e9-cd752ad3580b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:22.294605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0e63c-1dc2-4333-9586-3f3ed7917c93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:22.296165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0e63c-1dc2-4333-9586-3f3ed7917c93 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:22.318317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-40e0e63c-1dc2-4333-9586-3f3ed7917c93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:22.318317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:22.318317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:22.318317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:22.318317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:22.318317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:22.318317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:22.318317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:22.318317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:22.318317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:22.318317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:22.318317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:22.318317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:22.318317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:22.318317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:22.319455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:22.319455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:22.319455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:22.319455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:22.319455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:22.319455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:22.319455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:22.319455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:22.319455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:22.319455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:22.319455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:22.319455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:22.319455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:22.319455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:22.319455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:22.319455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:22.319455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:22.320737 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:22.320737 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:22.320737 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:22.320737 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:22.320737 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 337/703] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:22.326566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88293c9e-a24b-43b0-bb20-7c8f3cf1a8b1 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:22.326852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88293c9e-a24b-43b0-bb20-7c8f3cf1a8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:22.327121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88293c9e-a24b-43b0-bb20-7c8f3cf1a8b1 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:22.332950 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 338/704] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:22.334186 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88293c9e-a24b-43b0-bb20-7c8f3cf1a8b1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:22.394707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:22.434643 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3ab26-1c0c-4a83-96e9-cd752ad3580b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:22.482145 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56f214-09c1-45a6-8b4f-821122bc14ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:22.482752 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56f214-09c1-45a6-8b4f-821122bc14ad None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:22.482998 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56f214-09c1-45a6-8b4f-821122bc14ad None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:22.483319 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56f214-09c1-45a6-8b4f-821122bc14ad None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:22.484137 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc56f214-09c1-45a6-8b4f-821122bc14ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:22.484454 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc56f214-09c1-45a6-8b4f-821122bc14ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:22.485936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc56f214-09c1-45a6-8b4f-821122bc14ad None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:22.492212 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88293c9e-a24b-43b0-bb20-7c8f3cf1a8b1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:22.503518 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 367/705] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:22.504826 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e5678-51cf-4ab1-bade-1f70e67ab9ba None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:22.505125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e5678-51cf-4ab1-bade-1f70e67ab9ba None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:22.505403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e5678-51cf-4ab1-bade-1f70e67ab9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:22.531562 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-678e5678-51cf-4ab1-bade-1f70e67ab9ba None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:22.532981 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-678e5678-51cf-4ab1-bade-1f70e67ab9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:22.547037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc33035-1eba-4edf-852f-dc126f1077fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:22.548437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc33035-1eba-4edf-852f-dc126f1077fb None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:22.549008 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc33035-1eba-4edf-852f-dc126f1077fb None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:22.550332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc33035-1eba-4edf-852f-dc126f1077fb None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:22.551459 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc33035-1eba-4edf-852f-dc126f1077fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:22.552032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc33035-1eba-4edf-852f-dc126f1077fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:22.554461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc33035-1eba-4edf-852f-dc126f1077fb None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:22.572247 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 339/706] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:22.573842 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc33035-1eba-4edf-852f-dc126f1077fb None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:22.575546 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:22.626236 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 368/707] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:22.633570 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e5678-51cf-4ab1-bade-1f70e67ab9ba None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:22.722601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc33035-1eba-4edf-852f-dc126f1077fb None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:22.731338 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:22.760963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca865a22-f54f-421a-a60f-af575a520f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:22.761840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca865a22-f54f-421a-a60f-af575a520f06 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:22.762262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca865a22-f54f-421a-a60f-af575a520f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:22.762631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca865a22-f54f-421a-a60f-af575a520f06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:22.763663 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca865a22-f54f-421a-a60f-af575a520f06 None admin] RBAC: Adding query filter params (role.id=057d9335b3b1412ebfb6bfb27f02f96a, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=937bacab8c1c45c0876b8e61f6f3b859) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:22.764084 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca865a22-f54f-421a-a60f-af575a520f06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:22.765631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca865a22-f54f-421a-a60f-af575a520f06 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:22.772293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 340/708] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:22 2022] GET /identity/v3/role_assignments?user.id=937bacab8c1c45c0876b8e61f6f3b859&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=057d9335b3b1412ebfb6bfb27f02f96a => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:22.772971 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e5678-51cf-4ab1-bade-1f70e67ab9ba None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:22.773364 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca865a22-f54f-421a-a60f-af575a520f06 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:22.816987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf66e2-ba7f-4647-aa38-efe1ee713dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:22.818247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf66e2-ba7f-4647-aa38-efe1ee713dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:22.818735 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf66e2-ba7f-4647-aa38-efe1ee713dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:22.819351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf66e2-ba7f-4647-aa38-efe1ee713dc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:22.828451 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf66e2-ba7f-4647-aa38-efe1ee713dc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:22.830652 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf66e2-ba7f-4647-aa38-efe1ee713dc6 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:22.854031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-aacf66e2-ba7f-4647-aa38-efe1ee713dc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:22.854031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:22.854031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:22.854031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:22.854031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:22.854031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:22.854031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:22.854031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:22.854031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:22.854031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:22.854031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:22.854031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:22.854031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:22.854031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:22.854031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:22.855009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:22.855009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:22.855009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:22.855009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:22.855009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:22.855009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:22.855009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:22.855009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:22.855009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:22.855009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:22.855009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:22.855009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:22.855009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:22.855009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:22.855009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:22.855009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:22.855009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:22.856433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:22.856433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:22.856433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:22.856433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:22.856433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:22.856433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:22.856433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:22.856433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:22.856433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 369/709] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:22.863136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf66e2-ba7f-4647-aa38-efe1ee713dc6 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:22.923036 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:22.979634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca865a22-f54f-421a-a60f-af575a520f06 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:23.011220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf66e2-ba7f-4647-aa38-efe1ee713dc6 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:23.048570 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92138a0f-f05c-43e0-bfb0-68907859d5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:23.050120 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92138a0f-f05c-43e0-bfb0-68907859d5cc None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:23.050408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92138a0f-f05c-43e0-bfb0-68907859d5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:23.050680 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92138a0f-f05c-43e0-bfb0-68907859d5cc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:23.051606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92138a0f-f05c-43e0-bfb0-68907859d5cc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:23.053274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92138a0f-f05c-43e0-bfb0-68907859d5cc None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:23.059344 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03d852-cafe-4ffd-bab9-dbe37eb9f964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:23.060018 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03d852-cafe-4ffd-bab9-dbe37eb9f964 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:23.060303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03d852-cafe-4ffd-bab9-dbe37eb9f964 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:23.060581 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03d852-cafe-4ffd-bab9-dbe37eb9f964 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:23.061373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03d852-cafe-4ffd-bab9-dbe37eb9f964 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:23.061683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03d852-cafe-4ffd-bab9-dbe37eb9f964 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:23.063225 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03d852-cafe-4ffd-bab9-dbe37eb9f964 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:23.069887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-92138a0f-f05c-43e0-bfb0-68907859d5cc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:29:23.069887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:23.069887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:23.069887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:23.069887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:23.069887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:23.069887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:23.069887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:23.069887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:29:23.069887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:23.069887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:23.069887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:23.069887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:23.069887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:23.069887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:23.069887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:23.070880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:23.070880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:23.070880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:23.070880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:23.070880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:23.070880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:23.070880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:23.070880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:23.070880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:23.070880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:23.070880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:23.070880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:23.070880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:23.070880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:23.070880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:23.070880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:23.071700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:23.071700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:23.071700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:23.071700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:23.071700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:23.071700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:23.071700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:23.071700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:23.071700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:23.071700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:23.071700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:23.071700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:23.071700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:23.071700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:29:23.071700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:23.072469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 341/710] 10.222.0.52 () {56 vars in 1129 bytes} [Tue Jul 12 13:29:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:23.078373 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92138a0f-f05c-43e0-bfb0-68907859d5cc None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:23.081359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 370/711] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:23.090735 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03d852-cafe-4ffd-bab9-dbe37eb9f964 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:23.121888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:23.196989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:23.207739 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92138a0f-f05c-43e0-bfb0-68907859d5cc None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:23.215402 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03d852-cafe-4ffd-bab9-dbe37eb9f964 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:23.251892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4c322-c94f-4c2d-9685-3b5b1e94a4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:23.252207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fedd6-0efc-4930-b3af-0354e7779015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:23.252558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4c322-c94f-4c2d-9685-3b5b1e94a4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:23.252730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fedd6-0efc-4930-b3af-0354e7779015 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:23.252833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4c322-c94f-4c2d-9685-3b5b1e94a4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:23.253012 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fedd6-0efc-4930-b3af-0354e7779015 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:23.253110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4c322-c94f-4c2d-9685-3b5b1e94a4c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:23.253287 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fedd6-0efc-4930-b3af-0354e7779015 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:23.253925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4c322-c94f-4c2d-9685-3b5b1e94a4c4 None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=0d3915cde0d04bcfb9b5ce8c89af1e5b) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:23.254114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fedd6-0efc-4930-b3af-0354e7779015 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:23.254278 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4c322-c94f-4c2d-9685-3b5b1e94a4c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:23.254435 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fedd6-0efc-4930-b3af-0354e7779015 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:23.255740 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4c322-c94f-4c2d-9685-3b5b1e94a4c4 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:23.255847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fedd6-0efc-4930-b3af-0354e7779015 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:23.262533 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 342/713] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Jul 12 13:29:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:23.262707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 371/713] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:23 2022] GET /identity/v3/role_assignments?user.id=0d3915cde0d04bcfb9b5ce8c89af1e5b&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=864a6966630544e8bd68bcafba905c6f => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:23.263839 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e672b06-2963-4608-88ce-157a8b9f118b None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:23.264143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e672b06-2963-4608-88ce-157a8b9f118b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:23.264539 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e672b06-2963-4608-88ce-157a8b9f118b None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:23.266031 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 343/714] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:23.273982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ac5d9-106f-47e1-b9ba-7cf27439e565 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:23.273982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ac5d9-106f-47e1-b9ba-7cf27439e565 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:23.273982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ac5d9-106f-47e1-b9ba-7cf27439e565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:23.275547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e672b06-2963-4608-88ce-157a8b9f118b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:23.315063 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-236ac5d9-106f-47e1-b9ba-7cf27439e565 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:23.317686 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-236ac5d9-106f-47e1-b9ba-7cf27439e565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:23.427004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:23.433227 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e672b06-2963-4608-88ce-157a8b9f118b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:23.475502 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23519244-134f-4d6e-93b2-c27132f0dbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:23.476314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23519244-134f-4d6e-93b2-c27132f0dbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:23.476599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23519244-134f-4d6e-93b2-c27132f0dbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:23.476873 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23519244-134f-4d6e-93b2-c27132f0dbc9 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:23.477587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23519244-134f-4d6e-93b2-c27132f0dbc9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:23.477895 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23519244-134f-4d6e-93b2-c27132f0dbc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:23.479354 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23519244-134f-4d6e-93b2-c27132f0dbc9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:23.485949 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 344/715] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Jul 12 13:29:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:23.488243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 372/716] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:23.513510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23519244-134f-4d6e-93b2-c27132f0dbc9 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:23.654459 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23519244-134f-4d6e-93b2-c27132f0dbc9 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:23.708340 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7650902-a0c4-4784-ab2b-a73ac721ebfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:23.709168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7650902-a0c4-4784-ab2b-a73ac721ebfd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:23.709452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7650902-a0c4-4784-ab2b-a73ac721ebfd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:23.709727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7650902-a0c4-4784-ab2b-a73ac721ebfd None admin] PATH_INFO: `/v3/services/network` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:23.710438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7650902-a0c4-4784-ab2b-a73ac721ebfd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:23.711877 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7650902-a0c4-4784-ab2b-a73ac721ebfd None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:23.730918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-a7650902-a0c4-4784-ab2b-a73ac721ebfd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:29:23.730918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:23.730918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:23.730918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:23.730918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:23.730918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:23.730918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:23.730918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:23.730918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:29:23.730918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:23.730918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:23.730918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:23.730918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:23.730918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:23.730918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:23.730918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:23.730918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:23.732032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:23.732032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:23.732032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:23.732032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:23.732032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:23.732032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:23.732032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:23.732032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:23.732032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:23.732032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:23.732032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:23.732032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:23.732032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:23.732032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:23.732032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:23.732032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:23.733556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:23.733556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:23.733556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:23.733556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:23.733556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:23.733556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:23.733556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:23.733556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:23.733556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:23.733556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:23.733556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:23.733556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:23.733556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:29:23.733556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:23.733556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 345/717] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:23 2022] GET /identity/v3/services/network => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:23.739311 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ac5d9-106f-47e1-b9ba-7cf27439e565 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:23.859605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:23.881884 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ac5d9-106f-47e1-b9ba-7cf27439e565 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:23.919758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a849b17-dc8a-4519-bc06-90f78493e7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:23.920787 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a849b17-dc8a-4519-bc06-90f78493e7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:23.921303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a849b17-dc8a-4519-bc06-90f78493e7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:23.921782 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a849b17-dc8a-4519-bc06-90f78493e7f4 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:23.922719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a849b17-dc8a-4519-bc06-90f78493e7f4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:23.923233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a849b17-dc8a-4519-bc06-90f78493e7f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:23.924830 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a849b17-dc8a-4519-bc06-90f78493e7f4 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:23.930396 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 373/718] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jul 12 13:29:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:23.937990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7650902-a0c4-4784-ab2b-a73ac721ebfd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:23.996852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:24.102628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7650902-a0c4-4784-ab2b-a73ac721ebfd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:24.158716 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db2f68-4894-4f43-a1bd-ba3e7c2dd17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:24.159389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db2f68-4894-4f43-a1bd-ba3e7c2dd17e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:24.159678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db2f68-4894-4f43-a1bd-ba3e7c2dd17e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:24.159951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db2f68-4894-4f43-a1bd-ba3e7c2dd17e None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:24.160682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db2f68-4894-4f43-a1bd-ba3e7c2dd17e None admin] RBAC: Adding query filter params (type=network) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:24.160985 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db2f68-4894-4f43-a1bd-ba3e7c2dd17e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:24.162440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db2f68-4894-4f43-a1bd-ba3e7c2dd17e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:24.174614 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 346/719] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jul 12 13:29:23 2022] GET /identity/v3/services?type=network => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:24.661838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24042f0-5c62-4488-aaa9-45a33e04e5ed None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:24.662395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24042f0-5c62-4488-aaa9-45a33e04e5ed None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:24.662395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24042f0-5c62-4488-aaa9-45a33e04e5ed None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:24.664119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 374/720] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:24.670625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0bada-cbf3-46a6-b6a7-1c4b50dd2287 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:24.670959 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0bada-cbf3-46a6-b6a7-1c4b50dd2287 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:24.671176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0bada-cbf3-46a6-b6a7-1c4b50dd2287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:24.699117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-bed0bada-cbf3-46a6-b6a7-1c4b50dd2287 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:24.700370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bed0bada-cbf3-46a6-b6a7-1c4b50dd2287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:24.788638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 347/721] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:29:24.823617 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24042f0-5c62-4488-aaa9-45a33e04e5ed None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:24.963382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24042f0-5c62-4488-aaa9-45a33e04e5ed None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:25.019270 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df9d7c-1fb8-4fc0-a658-fe57dcee8bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:25.020206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df9d7c-1fb8-4fc0-a658-fe57dcee8bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.020488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df9d7c-1fb8-4fc0-a658-fe57dcee8bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.020761 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df9d7c-1fb8-4fc0-a658-fe57dcee8bc4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.036723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9df9d7c-1fb8-4fc0-a658-fe57dcee8bc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:25.044742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-e9df9d7c-1fb8-4fc0-a658-fe57dcee8bc4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:25.044742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:25.044742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:25.044742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:25.044742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:25.044742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:25.044742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:25.044742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:25.044742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:25.044742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:25.044742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:25.044742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:25.044742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:25.044742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:25.044742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:25.044742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:25.045723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:25.045723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:25.045723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:25.045723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:25.045723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:25.045723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:25.045723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:25.045723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:25.045723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:25.045723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:25.045723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:25.045723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:25.045723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:25.045723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:25.045723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:25.045723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:25.047228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:25.047228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:25.047228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:25.047228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:25.047228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:25.047228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:25.047228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:25.047228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:25.047228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:25.047228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:25.047228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:25.047228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:25.047228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 375/722] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:25.051970 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0bada-cbf3-46a6-b6a7-1c4b50dd2287 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:25.086386 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:25.184143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0bada-cbf3-46a6-b6a7-1c4b50dd2287 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:25.225617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea500919-2e9b-4b59-936a-c49875cfa694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:25.226233 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea500919-2e9b-4b59-936a-c49875cfa694 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.226510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea500919-2e9b-4b59-936a-c49875cfa694 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.226815 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea500919-2e9b-4b59-936a-c49875cfa694 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.227534 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea500919-2e9b-4b59-936a-c49875cfa694 None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:25.227836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea500919-2e9b-4b59-936a-c49875cfa694 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:25.229215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea500919-2e9b-4b59-936a-c49875cfa694 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:25.238032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 348/723] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:29:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:25.243688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df9d7c-1fb8-4fc0-a658-fe57dcee8bc4 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:25.255012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:25.391613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df9d7c-1fb8-4fc0-a658-fe57dcee8bc4 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:25.441249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b24102-530e-4e05-a1cc-5c29388c7677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:25.442469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b24102-530e-4e05-a1cc-5c29388c7677 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.442965 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b24102-530e-4e05-a1cc-5c29388c7677 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.443443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b24102-530e-4e05-a1cc-5c29388c7677 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.461889 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b24102-530e-4e05-a1cc-5c29388c7677 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:25.463788 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b24102-530e-4e05-a1cc-5c29388c7677 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:25.484814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-46b24102-530e-4e05-a1cc-5c29388c7677 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:25.484814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:25.484814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:25.484814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:25.484814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:25.484814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:25.484814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:25.484814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:25.484814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:25.484814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:25.484814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:25.484814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:25.484814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:25.484814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:25.484814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:25.485714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:25.485714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:25.485714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:25.485714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:25.485714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:25.485714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:25.485714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:25.485714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:25.485714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:25.485714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:25.485714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:25.485714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:25.485714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:25.485714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:25.485714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:25.485714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:25.485714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:25.486720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:25.486720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:25.486720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:25.486720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:25.488628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 376/724] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:25.494197 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea500919-2e9b-4b59-936a-c49875cfa694 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:25.586176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:25.609351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea500919-2e9b-4b59-936a-c49875cfa694 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:25.645521 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc461471-e184-4e2a-8de0-59033566b73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:25.646572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc461471-e184-4e2a-8de0-59033566b73a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.647088 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc461471-e184-4e2a-8de0-59033566b73a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.647581 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc461471-e184-4e2a-8de0-59033566b73a None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.648519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc461471-e184-4e2a-8de0-59033566b73a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:25.649029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc461471-e184-4e2a-8de0-59033566b73a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:25.650704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc461471-e184-4e2a-8de0-59033566b73a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:25.663651 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 349/725] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:25.671038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b24102-530e-4e05-a1cc-5c29388c7677 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:25.750919 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:25.800879 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571969-cded-47cf-9ec0-3912b84314ab None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.801226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571969-cded-47cf-9ec0-3912b84314ab None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.801437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571969-cded-47cf-9ec0-3912b84314ab None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.802970 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 350/726] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:29:25.808582 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95000b9c-9690-4102-a1d9-fe4ac7bd7e78 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.809181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95000b9c-9690-4102-a1d9-fe4ac7bd7e78 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.809662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95000b9c-9690-4102-a1d9-fe4ac7bd7e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.836333 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-95000b9c-9690-4102-a1d9-fe4ac7bd7e78 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:25.840761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-95000b9c-9690-4102-a1d9-fe4ac7bd7e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:25.850107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b24102-530e-4e05-a1cc-5c29388c7677 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:25.913880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1105a-ed4f-43d9-bb38-f6e73c2cccca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:25.914773 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1105a-ed4f-43d9-bb38-f6e73c2cccca None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.915059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1105a-ed4f-43d9-bb38-f6e73c2cccca None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.915337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1105a-ed4f-43d9-bb38-f6e73c2cccca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.923796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1105a-ed4f-43d9-bb38-f6e73c2cccca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:25.925257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1105a-ed4f-43d9-bb38-f6e73c2cccca None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:25.941208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-33f1105a-ed4f-43d9-bb38-f6e73c2cccca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:29:25.941208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:25.941208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:25.941208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:25.941208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:25.941208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:25.941208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:25.941208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:25.941208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:25.941208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:25.941208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:25.941208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:25.941208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:25.941208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:25.941208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:25.941208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:25.942220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:25.943621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:25.943621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:25.943621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:25.943621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:25.943621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:25.943621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:25.943621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:25.943621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:25.943621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:29:25.943621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:25.947239 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 377/727] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jul 12 13:29:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:25.948652 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cff93-400c-4179-9650-c8375e18622b None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.949002 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cff93-400c-4179-9650-c8375e18622b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.949766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cff93-400c-4179-9650-c8375e18622b None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.951555 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 378/728] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:25.952256 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cff93-400c-4179-9650-c8375e18622b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:25.958128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 351/729] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:29:25.962717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4520f-ce7c-4fc2-afa2-1daa234e5e0d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.963007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4520f-ce7c-4fc2-afa2-1daa234e5e0d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.963284 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4520f-ce7c-4fc2-afa2-1daa234e5e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.990841 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9ef4520f-ce7c-4fc2-afa2-1daa234e5e0d None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:25.992035 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9ef4520f-ce7c-4fc2-afa2-1daa234e5e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:26.069282 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 352/730] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:26.070500 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4520f-ce7c-4fc2-afa2-1daa234e5e0d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.078919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:26.119679 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cff93-400c-4179-9650-c8375e18622b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.141245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:26.159925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aac5d8-8f81-49b4-a993-1d69f2c556c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.160624 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aac5d8-8f81-49b4-a993-1d69f2c556c2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.160910 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aac5d8-8f81-49b4-a993-1d69f2c556c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.161181 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aac5d8-8f81-49b4-a993-1d69f2c556c2 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.162053 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aac5d8-8f81-49b4-a993-1d69f2c556c2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:26.162366 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aac5d8-8f81-49b4-a993-1d69f2c556c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:26.163830 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aac5d8-8f81-49b4-a993-1d69f2c556c2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:26.178526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4520f-ce7c-4fc2-afa2-1daa234e5e0d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.181474 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 379/731] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Jul 12 13:29:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:26.182109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aac5d8-8f81-49b4-a993-1d69f2c556c2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.223140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2205a778-f1a7-4fa4-9ea0-86d1e2ccafc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.223935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205a778-f1a7-4fa4-9ea0-86d1e2ccafc4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.224212 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205a778-f1a7-4fa4-9ea0-86d1e2ccafc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.224482 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205a778-f1a7-4fa4-9ea0-86d1e2ccafc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.236020 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2205a778-f1a7-4fa4-9ea0-86d1e2ccafc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:26.243792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-2205a778-f1a7-4fa4-9ea0-86d1e2ccafc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:29:26.243792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:26.243792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:26.243792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:26.243792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:26.243792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:26.243792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:26.243792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:26.243792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:26.243792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:26.243792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:26.243792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:26.243792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:26.243792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:26.243792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:26.243792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:26.244760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:26.244760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:26.244760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:26.244760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:26.244760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:26.244760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:26.244760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:26.244760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:26.244760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:26.244760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:26.244760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:26.244760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:26.244760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:26.244760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:26.244760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:26.244760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:26.245638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:26.245638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:26.245638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:26.245638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:26.245638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:26.245638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:26.245638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:26.245638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:26.245638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:26.245638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:26.245638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:29:26.245638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:26.245638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 353/732] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:29:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:26.247434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f36f9-f4ee-423a-9e6d-8b3303b20843 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.247434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f36f9-f4ee-423a-9e6d-8b3303b20843 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.247558 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f36f9-f4ee-423a-9e6d-8b3303b20843 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.253058 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 354/733] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:26.253058 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f36f9-f4ee-423a-9e6d-8b3303b20843 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.299640 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aac5d8-8f81-49b4-a993-1d69f2c556c2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.317255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:26.337974 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae174637-549c-41b7-9140-aeb5da57b630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.338704 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae174637-549c-41b7-9140-aeb5da57b630 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.338989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae174637-549c-41b7-9140-aeb5da57b630 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.339270 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae174637-549c-41b7-9140-aeb5da57b630 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.349990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae174637-549c-41b7-9140-aeb5da57b630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:26.358317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-ae174637-549c-41b7-9140-aeb5da57b630 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:29:26.358317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:26.358317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:26.358317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:26.358317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:26.358317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:26.358317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:26.358317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:26.358317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:26.358317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:26.358317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:26.358317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:26.358317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:26.358317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:26.358317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:26.358317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:26.359247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:26.359247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:26.359247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:26.359247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:26.359247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:26.359247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:26.359247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:26.359247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:26.359247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:26.359247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:26.359247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:26.359247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:26.359247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:26.359247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:26.359247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:26.359247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:26.360074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:26.360074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:26.360074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:26.360074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:26.360074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:26.360074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:26.360074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:26.360074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:26.360074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:26.360074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:26.360074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:29:26.360074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:26.360074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 380/734] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:29:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:26.361227 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae174637-549c-41b7-9140-aeb5da57b630 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.366668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f36f9-f4ee-423a-9e6d-8b3303b20843 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.399936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:26.402753 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ad00a-e05a-4542-91e2-4827dd2f0b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.403410 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ad00a-e05a-4542-91e2-4827dd2f0b16 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.403694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ad00a-e05a-4542-91e2-4827dd2f0b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.403970 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ad00a-e05a-4542-91e2-4827dd2f0b16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.469500 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae174637-549c-41b7-9140-aeb5da57b630 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.476997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ad00a-e05a-4542-91e2-4827dd2f0b16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:26.478516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ad00a-e05a-4542-91e2-4827dd2f0b16 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:26.513150 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238c058-1adf-47d6-9774-0e89e5ef9b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.513828 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238c058-1adf-47d6-9774-0e89e5ef9b21 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.514266 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238c058-1adf-47d6-9774-0e89e5ef9b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.514547 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238c058-1adf-47d6-9774-0e89e5ef9b21 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.515413 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238c058-1adf-47d6-9774-0e89e5ef9b21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:26.515724 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238c058-1adf-47d6-9774-0e89e5ef9b21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:26.517154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238c058-1adf-47d6-9774-0e89e5ef9b21 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:26.530118 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 381/735] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:29:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:26.531470 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2687e8-555c-4571-ba3b-694260ea51ee None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.531764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2687e8-555c-4571-ba3b-694260ea51ee None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.532040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2687e8-555c-4571-ba3b-694260ea51ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.549201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:26.549499 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8d2687e8-555c-4571-ba3b-694260ea51ee None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:26.550707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8d2687e8-555c-4571-ba3b-694260ea51ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:26.592952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 355/736] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:29:26.593712 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ad00a-e05a-4542-91e2-4827dd2f0b16 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.634043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 382/737] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:26.634043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2687e8-555c-4571-ba3b-694260ea51ee None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.669197 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:26.710883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ad00a-e05a-4542-91e2-4827dd2f0b16 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.749329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f817dd-fc8d-4ef4-a75f-432761241a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.750030 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f817dd-fc8d-4ef4-a75f-432761241a76 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.750321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f817dd-fc8d-4ef4-a75f-432761241a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.750599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f817dd-fc8d-4ef4-a75f-432761241a76 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.750700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2687e8-555c-4571-ba3b-694260ea51ee None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.751352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f817dd-fc8d-4ef4-a75f-432761241a76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:26.751967 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f817dd-fc8d-4ef4-a75f-432761241a76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:26.753346 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f817dd-fc8d-4ef4-a75f-432761241a76 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:26.762043 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 356/738] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:29:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:26.762865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f817dd-fc8d-4ef4-a75f-432761241a76 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.790013 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:26.793337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76303418-4c07-4c2e-9bc7-62130512788c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.794147 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76303418-4c07-4c2e-9bc7-62130512788c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.794436 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76303418-4c07-4c2e-9bc7-62130512788c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.794713 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76303418-4c07-4c2e-9bc7-62130512788c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.804886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76303418-4c07-4c2e-9bc7-62130512788c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:26.806369 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76303418-4c07-4c2e-9bc7-62130512788c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:26.823996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-76303418-4c07-4c2e-9bc7-62130512788c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:26.823996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:26.823996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:26.823996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:26.823996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:26.823996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:26.823996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:26.823996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:26.823996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:26.823996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:26.823996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:26.823996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:26.823996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:26.823996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:26.823996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:26.825070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:26.825070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:26.825070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:26.825070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:26.825070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:26.825070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:26.825070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:26.825070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:26.825070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:26.825070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:26.825070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:26.825070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:26.825070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:26.825070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:26.825070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:26.825070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:26.825070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:26.826043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:26.826043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:26.826043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:26.826043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:26.826043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 383/739] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:26.826043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76303418-4c07-4c2e-9bc7-62130512788c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.874993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f817dd-fc8d-4ef4-a75f-432761241a76 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.908223 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c01c8-dbbe-4c82-8ef3-ab7d646e4bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.909018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c01c8-dbbe-4c82-8ef3-ab7d646e4bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.909297 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c01c8-dbbe-4c82-8ef3-ab7d646e4bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.909571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c01c8-dbbe-4c82-8ef3-ab7d646e4bf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.914450 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:26.921053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c01c8-dbbe-4c82-8ef3-ab7d646e4bf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:26.922545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c01c8-dbbe-4c82-8ef3-ab7d646e4bf6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:26.934275 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76303418-4c07-4c2e-9bc7-62130512788c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.939669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-1e3c01c8-dbbe-4c82-8ef3-ab7d646e4bf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:26.939669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:26.939669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:26.939669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:26.939669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:26.939669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:26.939669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:26.939669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:26.939669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:26.939669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:26.939669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:26.939669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:26.939669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:26.939669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:26.939669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:26.940748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:26.940748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:26.940748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:26.940748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:26.940748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:26.940748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:26.940748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:26.940748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:26.940748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:26.940748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:26.940748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:26.940748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:26.940748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:26.940748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:26.940748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:26.940748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:26.940748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:26.946332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:26.946332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:26.946332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:26.946332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:26.946332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 357/740] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:26.946332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c01c8-dbbe-4c82-8ef3-ab7d646e4bf6 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.969867 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d95ed-254c-4d46-b80e-488ee1b8afac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.970580 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d95ed-254c-4d46-b80e-488ee1b8afac None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.970862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d95ed-254c-4d46-b80e-488ee1b8afac None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.971138 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d95ed-254c-4d46-b80e-488ee1b8afac None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.971856 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d95ed-254c-4d46-b80e-488ee1b8afac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:26.973232 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d95ed-254c-4d46-b80e-488ee1b8afac None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:26.988064 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 384/741] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jul 12 13:29:26 2022] POST /identity/v3/services => generated 251 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:26.989404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56377d-72a5-431e-9684-6bd174dc77ad None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.989690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56377d-72a5-431e-9684-6bd174dc77ad None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.989983 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56377d-72a5-431e-9684-6bd174dc77ad None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.995558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 385/742] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:26.996590 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56377d-72a5-431e-9684-6bd174dc77ad None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:27.025800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:27.053584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c01c8-dbbe-4c82-8ef3-ab7d646e4bf6 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:27.109241 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cd414-2a80-4da0-943d-b91f8c6d2d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:27.109543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cd414-2a80-4da0-943d-b91f8c6d2d95 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.109645 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cd414-2a80-4da0-943d-b91f8c6d2d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.109902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cd414-2a80-4da0-943d-b91f8c6d2d95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.120221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56377d-72a5-431e-9684-6bd174dc77ad None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:27.121389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4cd414-2a80-4da0-943d-b91f8c6d2d95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:27.123015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4cd414-2a80-4da0-943d-b91f8c6d2d95 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:27.142370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-ca4cd414-2a80-4da0-943d-b91f8c6d2d95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:27.142370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:27.142370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:27.142370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:27.142370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:27.142370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:27.142370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:27.142370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:27.142370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:27.142370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:27.142370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:27.142370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:27.142370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:27.142370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:27.142370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:27.143314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:27.143314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:27.143314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:27.143314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:27.143314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:27.143314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:27.143314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:27.143314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:27.143314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:27.143314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:27.143314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:27.143314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:27.143314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:27.143314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:27.143314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:27.143314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:27.143314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:27.144193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:27.144193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:27.144193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:27.144193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:27.144193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 358/743] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:27.145255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cd414-2a80-4da0-943d-b91f8c6d2d95 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:27.166194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706ded6-b0cf-4b42-9cb5-311b0aa6717b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:27.166667 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706ded6-b0cf-4b42-9cb5-311b0aa6717b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.167006 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706ded6-b0cf-4b42-9cb5-311b0aa6717b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.167291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706ded6-b0cf-4b42-9cb5-311b0aa6717b None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.168149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706ded6-b0cf-4b42-9cb5-311b0aa6717b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:27.168497 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706ded6-b0cf-4b42-9cb5-311b0aa6717b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:27.169922 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706ded6-b0cf-4b42-9cb5-311b0aa6717b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:27.179367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:27.187776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 386/744] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:27.187776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18693eed-49e3-461d-93c1-b7bb1185e37e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.187776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18693eed-49e3-461d-93c1-b7bb1185e37e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.187776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18693eed-49e3-461d-93c1-b7bb1185e37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.207322 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-18693eed-49e3-461d-93c1-b7bb1185e37e None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:27.208601 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-18693eed-49e3-461d-93c1-b7bb1185e37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:27.258550 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cd414-2a80-4da0-943d-b91f8c6d2d95 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:27.285678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:27.290965 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 387/745] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:29:27.291810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18693eed-49e3-461d-93c1-b7bb1185e37e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:27.305432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311c6bb-21ef-4c4d-bcc0-e4e6105c25cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:27.306547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311c6bb-21ef-4c4d-bcc0-e4e6105c25cc None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.307030 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311c6bb-21ef-4c4d-bcc0-e4e6105c25cc None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.307609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311c6bb-21ef-4c4d-bcc0-e4e6105c25cc None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.308625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311c6bb-21ef-4c4d-bcc0-e4e6105c25cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:27.309175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311c6bb-21ef-4c4d-bcc0-e4e6105c25cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:27.311151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311c6bb-21ef-4c4d-bcc0-e4e6105c25cc None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:27.324865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 359/746] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:27.325715 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311c6bb-21ef-4c4d-bcc0-e4e6105c25cc None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:27.411443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18693eed-49e3-461d-93c1-b7bb1185e37e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:27.428100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311c6bb-21ef-4c4d-bcc0-e4e6105c25cc None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:27.441803 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:27.449809 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ca78a-8f54-492e-8cd5-1ab02f61208d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:27.450514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ca78a-8f54-492e-8cd5-1ab02f61208d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.450785 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ca78a-8f54-492e-8cd5-1ab02f61208d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.451058 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ca78a-8f54-492e-8cd5-1ab02f61208d None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.451773 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ca78a-8f54-492e-8cd5-1ab02f61208d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:27.452077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ca78a-8f54-492e-8cd5-1ab02f61208d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:27.453733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ca78a-8f54-492e-8cd5-1ab02f61208d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:27.464468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b65c0-829a-46ef-b4a3-06145004b4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:27.465268 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b65c0-829a-46ef-b4a3-06145004b4ac None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.465553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b65c0-829a-46ef-b4a3-06145004b4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.465829 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b65c0-829a-46ef-b4a3-06145004b4ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.470592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 388/747] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:27.470592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ca78a-8f54-492e-8cd5-1ab02f61208d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:27.479497 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b65c0-829a-46ef-b4a3-06145004b4ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:27.479497 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b65c0-829a-46ef-b4a3-06145004b4ac None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:27.495239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-836b65c0-829a-46ef-b4a3-06145004b4ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:29:27.495239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:27.495239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:27.495239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:27.495239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:27.495239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:27.495239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:27.495239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:27.495239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:27.495239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:27.495239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:27.495239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:27.495239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:27.495239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:27.495239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:27.495239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:27.496771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:27.497703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:27.497703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:27.497703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:27.497703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:27.497703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:27.497703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:27.497703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:27.497703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:27.497703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:29:27.497703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:27.497703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 360/748] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jul 12 13:29:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:27.498973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b65c0-829a-46ef-b4a3-06145004b4ac None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:27.546390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:27.589488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ca78a-8f54-492e-8cd5-1ab02f61208d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:27.612834 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b65c0-829a-46ef-b4a3-06145004b4ac None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:27.629654 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82801b-eecf-4e4b-b781-e1b256e081af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:27.630594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82801b-eecf-4e4b-b781-e1b256e081af None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.630894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82801b-eecf-4e4b-b781-e1b256e081af None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.631176 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82801b-eecf-4e4b-b781-e1b256e081af None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.632289 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82801b-eecf-4e4b-b781-e1b256e081af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:27.633687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82801b-eecf-4e4b-b781-e1b256e081af None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:27.650307 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca0c71-8ce4-4a4e-bdb1-8b99603de4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:27.651100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca0c71-8ce4-4a4e-bdb1-8b99603de4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.662280 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 389/749] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jul 12 13:29:27 2022] POST /identity/v3/services => generated 240 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:27.662388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca0c71-8ce4-4a4e-bdb1-8b99603de4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.662388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca0c71-8ce4-4a4e-bdb1-8b99603de4f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.662573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82801b-eecf-4e4b-b781-e1b256e081af None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:27.662658 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca0c71-8ce4-4a4e-bdb1-8b99603de4f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:27.662658 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca0c71-8ce4-4a4e-bdb1-8b99603de4f3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:27.677769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-5eca0c71-8ce4-4a4e-bdb1-8b99603de4f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:27.677769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:27.677769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:27.677769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:27.677769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:27.677769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:27.677769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:27.677769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:27.677769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:27.677769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:27.677769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:27.677769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:27.677769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:27.677769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:27.677769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:27.678778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:27.678778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:27.678778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:27.678778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:27.678778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:27.678778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:27.678778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:27.678778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:27.678778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:27.678778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:27.678778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:27.678778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:27.678778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:27.678778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:27.678778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:27.678778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:27.678778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:27.679692 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:27.679692 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:27.679692 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:27.679692 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:27.679692 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:27.679692 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:27.679692 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:27.679692 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:27.684235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 361/750] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:27.685274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca0c71-8ce4-4a4e-bdb1-8b99603de4f3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:27.748486 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:27.812926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca0c71-8ce4-4a4e-bdb1-8b99603de4f3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:27.817053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:27.841196 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82801b-eecf-4e4b-b781-e1b256e081af None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:27.861747 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ba0f0-2c1b-4081-846f-e2040e1a09fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:27.862443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ba0f0-2c1b-4081-846f-e2040e1a09fe None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.862729 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ba0f0-2c1b-4081-846f-e2040e1a09fe None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.863003 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ba0f0-2c1b-4081-846f-e2040e1a09fe None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.863840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ba0f0-2c1b-4081-846f-e2040e1a09fe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:27.864151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ba0f0-2c1b-4081-846f-e2040e1a09fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:27.865736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ba0f0-2c1b-4081-846f-e2040e1a09fe None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:27.887396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 362/751] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jul 12 13:29:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:27.888236 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ba0f0-2c1b-4081-846f-e2040e1a09fe None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:27.894400 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb842a-39e8-4785-8eac-0d7e1ffe8399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:27.895213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb842a-39e8-4785-8eac-0d7e1ffe8399 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.895499 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb842a-39e8-4785-8eac-0d7e1ffe8399 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.895776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb842a-39e8-4785-8eac-0d7e1ffe8399 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.905022 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb842a-39e8-4785-8eac-0d7e1ffe8399 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:27.906550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb842a-39e8-4785-8eac-0d7e1ffe8399 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:27.922655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-55cb842a-39e8-4785-8eac-0d7e1ffe8399 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:29:27.922655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:27.922655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:27.922655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:27.922655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:27.922655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:27.922655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:27.922655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:27.922655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:27.922655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:27.922655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:27.922655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:27.922655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:27.922655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:27.922655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:27.922655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:27.923633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:27.925038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:27.925038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:27.925038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:27.925038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:27.925038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:27.925038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:27.925038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:27.925038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:27.925038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:29:27.925038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:27.925038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 390/752] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:29:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:27.925038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb842a-39e8-4785-8eac-0d7e1ffe8399 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:28.022068 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ba0f0-2c1b-4081-846f-e2040e1a09fe None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:28.037196 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb842a-39e8-4785-8eac-0d7e1ffe8399 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:28.052501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:28.062256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77edf4c5-05e5-4860-9686-d365866438ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:28.063200 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edf4c5-05e5-4860-9686-d365866438ef None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:28.063669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edf4c5-05e5-4860-9686-d365866438ef None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:28.064128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edf4c5-05e5-4860-9686-d365866438ef None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:28.065232 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77edf4c5-05e5-4860-9686-d365866438ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:28.065730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77edf4c5-05e5-4860-9686-d365866438ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:28.067509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77edf4c5-05e5-4860-9686-d365866438ef None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:28.080887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 363/753] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:28.082038 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77edf4c5-05e5-4860-9686-d365866438ef None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:28.085675 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e55f03-238e-4732-b980-325c9cd1a54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:28.086558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e55f03-238e-4732-b980-325c9cd1a54a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:28.086852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e55f03-238e-4732-b980-325c9cd1a54a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:28.087135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e55f03-238e-4732-b980-325c9cd1a54a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:28.100793 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e55f03-238e-4732-b980-325c9cd1a54a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:28.102464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e55f03-238e-4732-b980-325c9cd1a54a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:28.127313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-e2e55f03-238e-4732-b980-325c9cd1a54a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:28.127313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:28.127313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:28.127313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:28.127313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:28.127313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:28.127313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:28.127313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:28.127313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:28.127313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:28.127313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:28.127313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:28.127313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:28.127313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:28.127313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:28.128303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:28.128303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:28.128303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:28.128303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:28.128303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:28.128303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:28.128303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:28.128303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:28.128303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:28.128303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:28.128303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:28.128303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:28.128303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:28.128303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:28.128303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:28.128303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:28.128303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:28.129255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:28.129255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:28.129255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:28.129255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:28.129255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 391/754] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:28.130039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e55f03-238e-4732-b980-325c9cd1a54a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:28.133345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:28.199326 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77edf4c5-05e5-4860-9686-d365866438ef None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:28.237654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5d7ab-547a-44ad-bded-443d994cb9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:28.238780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5d7ab-547a-44ad-bded-443d994cb9bf None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:28.239289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5d7ab-547a-44ad-bded-443d994cb9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:28.239763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5d7ab-547a-44ad-bded-443d994cb9bf None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:28.240811 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5d7ab-547a-44ad-bded-443d994cb9bf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:28.241326 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5d7ab-547a-44ad-bded-443d994cb9bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:28.242998 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5d7ab-547a-44ad-bded-443d994cb9bf None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:28.259861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e55f03-238e-4732-b980-325c9cd1a54a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:28.265402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 364/755] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Jul 12 13:29:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:28.266846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5d7ab-547a-44ad-bded-443d994cb9bf None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:28.301168 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90b72b-5f30-4893-b87d-8dcab7cce96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:28.302002 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90b72b-5f30-4893-b87d-8dcab7cce96e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:28.302410 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90b72b-5f30-4893-b87d-8dcab7cce96e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:28.302760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90b72b-5f30-4893-b87d-8dcab7cce96e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:28.303707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90b72b-5f30-4893-b87d-8dcab7cce96e None admin] RBAC: Adding query filter params (role.id=057d9335b3b1412ebfb6bfb27f02f96a, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=dedf564ba5744a1fa70bb78222d26d5b) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:28.304089 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90b72b-5f30-4893-b87d-8dcab7cce96e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:28.305836 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90b72b-5f30-4893-b87d-8dcab7cce96e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:28.318544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 392/756] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:28 2022] GET /identity/v3/role_assignments?user.id=dedf564ba5744a1fa70bb78222d26d5b&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=057d9335b3b1412ebfb6bfb27f02f96a => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:28.319602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90b72b-5f30-4893-b87d-8dcab7cce96e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:28.366339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:28.390674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5d7ab-547a-44ad-bded-443d994cb9bf None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:28.393307 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:28.432267 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bdabf-30eb-40eb-92c4-1c32324237c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:28.433516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bdabf-30eb-40eb-92c4-1c32324237c9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:28.434024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bdabf-30eb-40eb-92c4-1c32324237c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:28.434475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bdabf-30eb-40eb-92c4-1c32324237c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:28.435411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bdabf-30eb-40eb-92c4-1c32324237c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:28.435920 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bdabf-30eb-40eb-92c4-1c32324237c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:28.437518 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bdabf-30eb-40eb-92c4-1c32324237c9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:28.452279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 365/757] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:28.459042 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bdabf-30eb-40eb-92c4-1c32324237c9 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:28.560032 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90b72b-5f30-4893-b87d-8dcab7cce96e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:28.621155 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bdabf-30eb-40eb-92c4-1c32324237c9 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:28.664882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360eeec-83d0-4b0a-bdae-8009244a7ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:28.665691 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360eeec-83d0-4b0a-bdae-8009244a7ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:28.665990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360eeec-83d0-4b0a-bdae-8009244a7ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:28.666289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360eeec-83d0-4b0a-bdae-8009244a7ef2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:28.674983 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0360eeec-83d0-4b0a-bdae-8009244a7ef2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:28.676585 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0360eeec-83d0-4b0a-bdae-8009244a7ef2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:28.687287 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1d73a-ae5a-4a8a-b101-09d30b4ac6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:28.688276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1d73a-ae5a-4a8a-b101-09d30b4ac6cc None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:28.688824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1d73a-ae5a-4a8a-b101-09d30b4ac6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:28.689306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1d73a-ae5a-4a8a-b101-09d30b4ac6cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:28.694513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-0360eeec-83d0-4b0a-bdae-8009244a7ef2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:28.694513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:28.694513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:28.694513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:28.694513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:28.694513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:28.694513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:28.694513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:28.694513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:28.694513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:28.694513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:28.694513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:28.694513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:28.694513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:28.694513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:28.695662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:28.695662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:28.695662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:28.695662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:28.695662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:28.695662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:28.695662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:28.695662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:28.695662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:28.695662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:28.695662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:28.695662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:28.695662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:28.695662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:28.695662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:28.695662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:28.695662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:28.696674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:28.696674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:28.696674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:28.696674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:28.696674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:28.696674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:28.696674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:28.696674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:28.696674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:28.696674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 366/758] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:28.701113 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360eeec-83d0-4b0a-bdae-8009244a7ef2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:28.819555 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1d73a-ae5a-4a8a-b101-09d30b4ac6cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:28.824261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1d73a-ae5a-4a8a-b101-09d30b4ac6cc None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:28.878007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:28.890087 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360eeec-83d0-4b0a-bdae-8009244a7ef2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:28.929173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30832474-b179-43f7-8192-5fc3c7950051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:28.929838 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30832474-b179-43f7-8192-5fc3c7950051 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:28.930150 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30832474-b179-43f7-8192-5fc3c7950051 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:28.930427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30832474-b179-43f7-8192-5fc3c7950051 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:28.931215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30832474-b179-43f7-8192-5fc3c7950051 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:28.931524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30832474-b179-43f7-8192-5fc3c7950051 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:28.932972 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30832474-b179-43f7-8192-5fc3c7950051 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:28.963491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 367/759] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:28.972955 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30832474-b179-43f7-8192-5fc3c7950051 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:29.006863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 393/760] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:29:29.018020 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1d73a-ae5a-4a8a-b101-09d30b4ac6cc None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:29.101689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:29.121223 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1d73a-ae5a-4a8a-b101-09d30b4ac6cc None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:29.150599 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:29.154825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30832474-b179-43f7-8192-5fc3c7950051 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:29.163109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b1d9e-6575-49ec-9987-2cc1453e7b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:29.163910 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b1d9e-6575-49ec-9987-2cc1453e7b76 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:29.164191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b1d9e-6575-49ec-9987-2cc1453e7b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:29.164466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b1d9e-6575-49ec-9987-2cc1453e7b76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:29.175955 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b1d9e-6575-49ec-9987-2cc1453e7b76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:29.177423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b1d9e-6575-49ec-9987-2cc1453e7b76 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:29.190968 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8459f-06d8-4442-a475-83860f268a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:29.192117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8459f-06d8-4442-a475-83860f268a58 None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:29.192610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8459f-06d8-4442-a475-83860f268a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:29.193065 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8459f-06d8-4442-a475-83860f268a58 None admin] PATH_INFO: `/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/0d3915cde0d04bcfb9b5ce8c89af1e5b/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:29.195243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8459f-06d8-4442-a475-83860f268a58 None admin] RBAC: Authorizing `identity:create_grant(project_id=36aa0326fd0545dface07a9ddc308ccd, user_id=0d3915cde0d04bcfb9b5ce8c89af1e5b, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:29.196598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-f36b1d9e-6575-49ec-9987-2cc1453e7b76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:29.196598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:29.196598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:29.196598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:29.196598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:29.196598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:29.196598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:29.196598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:29.196598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:29.196598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:29.196598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:29.196598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:29.196598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:29.196598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:29.196598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:29.197485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:29.197485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:29.197485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:29.197485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:29.197485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:29.197485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:29.197485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:29.197485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:29.197485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:29.197485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:29.197485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:29.197485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:29.197485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:29.197485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:29.197485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:29.197485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:29.197485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:29.198421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:29.198421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:29.198421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:29.198421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:29.200054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8459f-06d8-4442-a475-83860f268a58 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:29.203070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 394/761] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:29.204528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1555b3-58da-449b-bb50-a90742b2ad39 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:29.204819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1555b3-58da-449b-bb50-a90742b2ad39 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:29.205150 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1555b3-58da-449b-bb50-a90742b2ad39 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:29.206707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 395/762] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:29.207558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1555b3-58da-449b-bb50-a90742b2ad39 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:29.217881 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 368/763] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:29:28 2022] PUT /identity/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/0d3915cde0d04bcfb9b5ce8c89af1e5b/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:29.219584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c676047-c444-4662-961e-484b6bd691da None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:29.220120 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c676047-c444-4662-961e-484b6bd691da None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:29.220639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c676047-c444-4662-961e-484b6bd691da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:29.240351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1c676047-c444-4662-961e-484b6bd691da None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:29.242035 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1c676047-c444-4662-961e-484b6bd691da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:29.323474 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 369/764] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:29.335029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1555b3-58da-449b-bb50-a90742b2ad39 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:29.386556 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69d2c4-8192-4d9c-a449-a682d6ff8f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:29.387223 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69d2c4-8192-4d9c-a449-a682d6ff8f80 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:29.387555 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69d2c4-8192-4d9c-a449-a682d6ff8f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:29.387832 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69d2c4-8192-4d9c-a449-a682d6ff8f80 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:29.388579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69d2c4-8192-4d9c-a449-a682d6ff8f80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:29.388887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69d2c4-8192-4d9c-a449-a682d6ff8f80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:29.390391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69d2c4-8192-4d9c-a449-a682d6ff8f80 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:29.395990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c676047-c444-4662-961e-484b6bd691da None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:29.404050 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 396/765] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:29.411541 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69d2c4-8192-4d9c-a449-a682d6ff8f80 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:29.463312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:29.473346 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:29.533271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c676047-c444-4662-961e-484b6bd691da None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:29.549843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69d2c4-8192-4d9c-a449-a682d6ff8f80 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:29.573942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cda21b-28ff-42a2-8ce4-ed1896d78909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:29.574785 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cda21b-28ff-42a2-8ce4-ed1896d78909 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:29.575095 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cda21b-28ff-42a2-8ce4-ed1896d78909 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:29.575443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cda21b-28ff-42a2-8ce4-ed1896d78909 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:29.576111 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cda21b-28ff-42a2-8ce4-ed1896d78909 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:29.577482 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cda21b-28ff-42a2-8ce4-ed1896d78909 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:29.590217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bebf8-e3c7-42b7-bca3-6a0ccf4752f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:29.591022 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bebf8-e3c7-42b7-bca3-6a0ccf4752f4 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:29.591305 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bebf8-e3c7-42b7-bca3-6a0ccf4752f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:29.591578 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bebf8-e3c7-42b7-bca3-6a0ccf4752f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:29.599157 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bebf8-e3c7-42b7-bca3-6a0ccf4752f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:29.600670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bebf8-e3c7-42b7-bca3-6a0ccf4752f4 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:29.611251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-90cda21b-28ff-42a2-8ce4-ed1896d78909 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:29:29.611251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:29.611251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:29.611251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:29.611251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:29.611251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:29.611251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:29.611251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:29.611251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:29:29.611251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:29.611251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:29.611251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:29.611251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:29.611251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:29.611251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:29.611251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:29.612209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:29.612209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:29.612209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:29.612209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:29.612209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:29.612209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:29.612209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:29.612209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:29.612209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:29.612209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:29.612209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:29.612209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:29.612209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:29.612209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:29.612209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:29.612209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:29.613128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:29.613128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:29.613128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:29.613128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:29.613128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:29.613128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:29.613128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:29.613128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:29.613128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:29.613128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:29.613128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:29.613128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:29.613128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:29.613128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:29:29.613128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:29.613128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 370/766] 10.222.0.52 () {56 vars in 1129 bytes} [Tue Jul 12 13:29:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:29.616044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-508bebf8-e3c7-42b7-bca3-6a0ccf4752f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:29.616044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:29.616044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:29.616044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:29.616044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:29.616044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:29.616044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:29.616044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:29.616044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:29.616044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:29.616044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:29.616044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:29.616044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:29.616044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:29.616044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:29.618073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:29.618073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:29.618073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:29.618073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:29.618073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:29.618073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:29.618073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:29.618073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:29.618073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:29.618073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:29.618073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:29.618073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:29.618073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:29.618073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:29.618073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:29.618073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:29.618073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:29.620250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:29.620250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:29.620250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:29.620250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:29.620250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:29.620250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:29.620250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:29.620250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:29.620250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 397/767] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:29.624235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cda21b-28ff-42a2-8ce4-ed1896d78909 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:29.624449 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bebf8-e3c7-42b7-bca3-6a0ccf4752f4 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:29.728578 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:29.737270 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bebf8-e3c7-42b7-bca3-6a0ccf4752f4 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:29.772847 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e3924-edd0-4f48-a6a6-c1c9a19fbe42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:29.773560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e3924-edd0-4f48-a6a6-c1c9a19fbe42 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:29.773847 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e3924-edd0-4f48-a6a6-c1c9a19fbe42 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:29.774186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e3924-edd0-4f48-a6a6-c1c9a19fbe42 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:29.774981 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e3924-edd0-4f48-a6a6-c1c9a19fbe42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:29.775291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e3924-edd0-4f48-a6a6-c1c9a19fbe42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:29.776787 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e3924-edd0-4f48-a6a6-c1c9a19fbe42 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:29.783823 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cda21b-28ff-42a2-8ce4-ed1896d78909 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:29.789669 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 398/768] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:29.796143 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e3924-edd0-4f48-a6a6-c1c9a19fbe42 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:29.799825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:29.823494 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98064235-be00-4b26-be93-30d1e2619de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:29.824381 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98064235-be00-4b26-be93-30d1e2619de6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:29.824666 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98064235-be00-4b26-be93-30d1e2619de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:29.825052 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98064235-be00-4b26-be93-30d1e2619de6 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:29.825767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98064235-be00-4b26-be93-30d1e2619de6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:29.826116 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98064235-be00-4b26-be93-30d1e2619de6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:29.827617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98064235-be00-4b26-be93-30d1e2619de6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:29.837150 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 371/769] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Jul 12 13:29:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:29.838916 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98064235-be00-4b26-be93-30d1e2619de6 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:29.953448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98064235-be00-4b26-be93-30d1e2619de6 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:29.991059 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972cfc9e-dbca-4875-9a3e-c6d92e485e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:29.991642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e3924-edd0-4f48-a6a6-c1c9a19fbe42 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:29.992169 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cfc9e-dbca-4875-9a3e-c6d92e485e02 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:29.992169 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cfc9e-dbca-4875-9a3e-c6d92e485e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:29.992169 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cfc9e-dbca-4875-9a3e-c6d92e485e02 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:29.992891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972cfc9e-dbca-4875-9a3e-c6d92e485e02 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:29.993378 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972cfc9e-dbca-4875-9a3e-c6d92e485e02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:29.994792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972cfc9e-dbca-4875-9a3e-c6d92e485e02 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:30.004185 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 372/770] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Jul 12 13:29:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:30.006639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972cfc9e-dbca-4875-9a3e-c6d92e485e02 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:30.053979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:30.095642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42a94d-783d-46d0-aafc-66ed222c30d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:30.096317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42a94d-783d-46d0-aafc-66ed222c30d0 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:30.096602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42a94d-783d-46d0-aafc-66ed222c30d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:30.096875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42a94d-783d-46d0-aafc-66ed222c30d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:30.097710 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42a94d-783d-46d0-aafc-66ed222c30d0 None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=937bacab8c1c45c0876b8e61f6f3b859) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:30.098043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42a94d-783d-46d0-aafc-66ed222c30d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:30.099498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42a94d-783d-46d0-aafc-66ed222c30d0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:30.115447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 399/771] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:29 2022] GET /identity/v3/role_assignments?user.id=937bacab8c1c45c0876b8e61f6f3b859&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=864a6966630544e8bd68bcafba905c6f => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:30.117103 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f4761-0217-4291-8c38-394daa7637f1 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:30.117399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f4761-0217-4291-8c38-394daa7637f1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:30.117680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f4761-0217-4291-8c38-394daa7637f1 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:30.119321 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 400/772] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:30.128738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972cfc9e-dbca-4875-9a3e-c6d92e485e02 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:30.131794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db603fb6-f8d2-4404-af4a-31d7d82bfd60 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:30.132260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db603fb6-f8d2-4404-af4a-31d7d82bfd60 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:30.132788 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db603fb6-f8d2-4404-af4a-31d7d82bfd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:30.155315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-db603fb6-f8d2-4404-af4a-31d7d82bfd60 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:30.156616 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-db603fb6-f8d2-4404-af4a-31d7d82bfd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:30.174094 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:30.259905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84be04d-0901-4797-b35f-619d64cb8518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:30.260509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84be04d-0901-4797-b35f-619d64cb8518 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:30.260797 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84be04d-0901-4797-b35f-619d64cb8518 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:30.261085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84be04d-0901-4797-b35f-619d64cb8518 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:30.262430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84be04d-0901-4797-b35f-619d64cb8518 None admin] RBAC: Adding query filter params (interface=public, service_id=215a92e350f14de2ba7b17beed573390, region_id=RegionOne) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:30.262818 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84be04d-0901-4797-b35f-619d64cb8518 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:30.264249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84be04d-0901-4797-b35f-619d64cb8518 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:30.271842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 401/773] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:29:30.277831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 373/774] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Jul 12 13:29:30 2022] GET /identity/v3/endpoints?service_id=215a92e350f14de2ba7b17beed573390&interface=public®ion_id=RegionOne => generated 194 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:30.279157 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db603fb6-f8d2-4404-af4a-31d7d82bfd60 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:30.322922 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84be04d-0901-4797-b35f-619d64cb8518 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:30.487858 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db603fb6-f8d2-4404-af4a-31d7d82bfd60 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:30.526440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84be04d-0901-4797-b35f-619d64cb8518 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:30.540529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:30.571436 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d884a3-2e87-499c-b748-e12ae99d6955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:30.572461 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d884a3-2e87-499c-b748-e12ae99d6955 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:30.572933 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d884a3-2e87-499c-b748-e12ae99d6955 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:30.573412 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d884a3-2e87-499c-b748-e12ae99d6955 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:30.574870 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d884a3-2e87-499c-b748-e12ae99d6955 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:30.577949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d884a3-2e87-499c-b748-e12ae99d6955 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:30.588716 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 402/775] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jul 12 13:29:30 2022] GET /identity/v3/services => generated 772 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:30.596326 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa296dc7-7aad-4f79-ba3c-6964e8d0926e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:30.597468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa296dc7-7aad-4f79-ba3c-6964e8d0926e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:30.597940 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa296dc7-7aad-4f79-ba3c-6964e8d0926e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:30.598432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa296dc7-7aad-4f79-ba3c-6964e8d0926e None admin] PATH_INFO: `/v3/services/network` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:30.599314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa296dc7-7aad-4f79-ba3c-6964e8d0926e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:30.600939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa296dc7-7aad-4f79-ba3c-6964e8d0926e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:30.614368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-fa296dc7-7aad-4f79-ba3c-6964e8d0926e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:29:30.614368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:30.614368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:30.614368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:30.614368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:30.614368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:30.614368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:30.614368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:30.614368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:29:30.614368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:30.614368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:30.614368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:30.614368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:30.614368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:30.614368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:30.614368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:30.614368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:30.615432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:30.615432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:30.615432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:30.615432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:30.615432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:30.615432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:30.615432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:30.615432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:30.615432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:30.615432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:30.615432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:30.615432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:30.615432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:30.615432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:30.615432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:30.615432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:30.616279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:30.616279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:30.616279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:30.616279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:30.616279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:30.616279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:30.616279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:30.616279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:30.616279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:30.616279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:30.616279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:30.616279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:30.616279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:29:30.616279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:30.624563 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 374/776] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:30 2022] GET /identity/v3/services/network => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:30.626239 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d884a3-2e87-499c-b748-e12ae99d6955 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:30.643695 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:30.744076 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d884a3-2e87-499c-b748-e12ae99d6955 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:30.789833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00736424-10a5-4c93-8a58-251ef69f75ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:30.790479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00736424-10a5-4c93-8a58-251ef69f75ac None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:30.790771 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00736424-10a5-4c93-8a58-251ef69f75ac None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:30.791044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00736424-10a5-4c93-8a58-251ef69f75ac None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:30.791656 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a990c80-0c96-49ce-a085-061489831518 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:30.791778 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00736424-10a5-4c93-8a58-251ef69f75ac None admin] RBAC: Adding query filter params (name=network) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:30.792063 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00736424-10a5-4c93-8a58-251ef69f75ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:30.793014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a990c80-0c96-49ce-a085-061489831518 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:30.793507 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a990c80-0c96-49ce-a085-061489831518 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:30.794069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00736424-10a5-4c93-8a58-251ef69f75ac None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:30.799659 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 403/778] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jul 12 13:29:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:30.799993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 375/778] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:30.800980 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b6259-d644-4b68-bd3c-d70bc81bcd5e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:30.801266 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b6259-d644-4b68-bd3c-d70bc81bcd5e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:30.801544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b6259-d644-4b68-bd3c-d70bc81bcd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:30.804802 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a990c80-0c96-49ce-a085-061489831518 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:30.819602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-417b6259-d644-4b68-bd3c-d70bc81bcd5e None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:30.820845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-417b6259-d644-4b68-bd3c-d70bc81bcd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:30.897730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 404/779] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:30.921320 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b6259-d644-4b68-bd3c-d70bc81bcd5e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:30.936406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:30.978736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a990c80-0c96-49ce-a085-061489831518 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:31.050764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacf5ea-578e-41da-abed-8ad4ac841590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:31.051954 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacf5ea-578e-41da-abed-8ad4ac841590 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:31.052140 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b6259-d644-4b68-bd3c-d70bc81bcd5e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:31.052572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacf5ea-578e-41da-abed-8ad4ac841590 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:31.052917 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacf5ea-578e-41da-abed-8ad4ac841590 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:31.053938 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacf5ea-578e-41da-abed-8ad4ac841590 None admin] RBAC: Adding query filter params (type=network) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:31.054278 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacf5ea-578e-41da-abed-8ad4ac841590 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:31.055672 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacf5ea-578e-41da-abed-8ad4ac841590 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:31.072493 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 376/780] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jul 12 13:29:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:31.078081 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacf5ea-578e-41da-abed-8ad4ac841590 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:31.099868 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a3b44-df7e-463b-8dfd-5b667e1348ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:31.100677 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a3b44-df7e-463b-8dfd-5b667e1348ce None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:31.100963 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a3b44-df7e-463b-8dfd-5b667e1348ce None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:31.101236 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a3b44-df7e-463b-8dfd-5b667e1348ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:31.127505 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a3b44-df7e-463b-8dfd-5b667e1348ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:31.137244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-ab9a3b44-df7e-463b-8dfd-5b667e1348ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:31.137244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:31.137244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:31.137244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:31.137244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:31.137244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:31.137244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:31.137244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:31.137244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:31.137244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:31.137244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:31.137244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:31.137244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:31.137244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:31.137244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:31.137244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:31.138251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:31.138251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:31.138251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:31.138251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:31.138251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:31.138251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:31.138251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:31.138251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:31.138251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:31.138251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:31.138251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:31.138251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:31.138251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:31.138251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:31.138251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:31.138251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:31.139110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:31.139110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:31.139110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:31.139110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:31.139110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:31.139110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:31.139110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:31.139110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:31.139110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:31.139110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:31.139110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:31.139110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:31.139110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 405/781] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:31.144246 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a3b44-df7e-463b-8dfd-5b667e1348ce None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:31.151216 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:31.221198 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacf5ea-578e-41da-abed-8ad4ac841590 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:31.256559 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3302c-af86-4559-a123-4b9d4a348e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:31.257228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3302c-af86-4559-a123-4b9d4a348e36 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:31.257522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3302c-af86-4559-a123-4b9d4a348e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:31.257800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3302c-af86-4559-a123-4b9d4a348e36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:31.258626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3302c-af86-4559-a123-4b9d4a348e36 None admin] RBAC: Adding query filter params (interface=public, service_id=dbd1f7348f7e4643a4b73867db06075a, region_id=RegionOne) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:31.259186 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3302c-af86-4559-a123-4b9d4a348e36 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:31.260761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3302c-af86-4559-a123-4b9d4a348e36 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:31.266926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 377/782] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Jul 12 13:29:31 2022] GET /identity/v3/endpoints?service_id=dbd1f7348f7e4643a4b73867db06075a&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:31.272226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3302c-af86-4559-a123-4b9d4a348e36 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:31.303781 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:31.343377 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a3b44-df7e-463b-8dfd-5b667e1348ce None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:31.380886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e138a3b-367e-4680-abfe-098c8a6353b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:31.381558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e138a3b-367e-4680-abfe-098c8a6353b6 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:31.381847 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e138a3b-367e-4680-abfe-098c8a6353b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:31.382156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e138a3b-367e-4680-abfe-098c8a6353b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:31.382909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e138a3b-367e-4680-abfe-098c8a6353b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:31.385996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e138a3b-367e-4680-abfe-098c8a6353b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:31.385996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e138a3b-367e-4680-abfe-098c8a6353b6 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:31.394287 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 406/783] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:29:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:31.407715 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e138a3b-367e-4680-abfe-098c8a6353b6 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:31.446378 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:31.448555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3302c-af86-4559-a123-4b9d4a348e36 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:31.485522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc917a-0f7b-4149-a439-201a52b9a85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:31.487640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc917a-0f7b-4149-a439-201a52b9a85e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:31.488592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc917a-0f7b-4149-a439-201a52b9a85e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:31.494660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc917a-0f7b-4149-a439-201a52b9a85e None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:31.495452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc917a-0f7b-4149-a439-201a52b9a85e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:31.497264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc917a-0f7b-4149-a439-201a52b9a85e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:31.504821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 378/784] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jul 12 13:29:31 2022] GET /identity/v3/services => generated 772 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:31.601999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e138a3b-367e-4680-abfe-098c8a6353b6 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:31.642741 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:31.654160 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31b948-6f54-4702-bd64-4b6b8795fe69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:31.654995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31b948-6f54-4702-bd64-4b6b8795fe69 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:31.655277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31b948-6f54-4702-bd64-4b6b8795fe69 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:31.655550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31b948-6f54-4702-bd64-4b6b8795fe69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:31.672265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31b948-6f54-4702-bd64-4b6b8795fe69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:31.673758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31b948-6f54-4702-bd64-4b6b8795fe69 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:31.698277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-1c31b948-6f54-4702-bd64-4b6b8795fe69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:31.698277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:31.698277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:31.698277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:31.698277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:31.698277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:31.698277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:31.698277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:31.698277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:31.698277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:31.698277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:31.698277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:31.698277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:31.698277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:31.698277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:31.699760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:31.699760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:31.699760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:31.699760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:31.699760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:31.699760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:31.699760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:31.699760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:31.699760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:31.699760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:31.699760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:31.699760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:31.699760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:31.699760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:31.699760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:31.699760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:31.699760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:31.700634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:31.700634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:31.700634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:31.700634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:31.700634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 407/785] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:31.705687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc917a-0f7b-4149-a439-201a52b9a85e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:31.828004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc917a-0f7b-4149-a439-201a52b9a85e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:31.870738 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a2c2c-e739-45ca-b0dc-a3270d457afb None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:31.871046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a2c2c-e739-45ca-b0dc-a3270d457afb None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:31.871206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a2c2c-e739-45ca-b0dc-a3270d457afb None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:31.872890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 408/786] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:31.879388 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fc065-2945-497e-9cf1-b238a9acf252 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:31.879815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fc065-2945-497e-9cf1-b238a9acf252 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:31.880227 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fc065-2945-497e-9cf1-b238a9acf252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:31.885092 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c7355-81f3-4720-a7ea-f0126ad4d053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:31.886599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c7355-81f3-4720-a7ea-f0126ad4d053 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:31.887089 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c7355-81f3-4720-a7ea-f0126ad4d053 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:31.887561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c7355-81f3-4720-a7ea-f0126ad4d053 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:31.888508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c7355-81f3-4720-a7ea-f0126ad4d053 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:31.889015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c7355-81f3-4720-a7ea-f0126ad4d053 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:31.890640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c7355-81f3-4720-a7ea-f0126ad4d053 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:31.904300 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 379/787] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:31.922009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d11fc065-2945-497e-9cf1-b238a9acf252 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:31.922602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c7355-81f3-4720-a7ea-f0126ad4d053 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:31.923231 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d11fc065-2945-497e-9cf1-b238a9acf252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:31.949883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:32.038256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c7355-81f3-4720-a7ea-f0126ad4d053 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:32.059890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 409/788] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:29:32.095828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00446258-1627-4175-bf3c-6094fac7fc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:32.097031 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00446258-1627-4175-bf3c-6094fac7fc45 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:32.097518 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00446258-1627-4175-bf3c-6094fac7fc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:32.098022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00446258-1627-4175-bf3c-6094fac7fc45 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:32.111918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00446258-1627-4175-bf3c-6094fac7fc45 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:32.113738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00446258-1627-4175-bf3c-6094fac7fc45 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:32.127104 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fc065-2945-497e-9cf1-b238a9acf252 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:32.148168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-00446258-1627-4175-bf3c-6094fac7fc45 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:29:32.148168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:32.148168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:32.148168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:32.148168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:32.148168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:32.148168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:32.148168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:32.148168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:32.148168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:32.148168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:32.148168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:32.148168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:32.148168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:32.148168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:32.148168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:32.149176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:32.150127 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:32.150127 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:32.150127 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:32.150127 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:32.150127 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:32.150127 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:32.150127 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:32.150127 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:32.150127 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:29:32.150127 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:32.153424 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 380/789] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jul 12 13:29:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:32.158597 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00446258-1627-4175-bf3c-6094fac7fc45 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:32.252890 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:32.275427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00446258-1627-4175-bf3c-6094fac7fc45 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:32.316294 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28affbbc-97da-4690-bfc9-59332c86cc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:32.317334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28affbbc-97da-4690-bfc9-59332c86cc71 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:32.317851 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28affbbc-97da-4690-bfc9-59332c86cc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:32.318390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28affbbc-97da-4690-bfc9-59332c86cc71 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:32.319430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28affbbc-97da-4690-bfc9-59332c86cc71 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:32.320453 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28affbbc-97da-4690-bfc9-59332c86cc71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:32.322142 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28affbbc-97da-4690-bfc9-59332c86cc71 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:32.342537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 381/790] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Jul 12 13:29:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:32.348313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28affbbc-97da-4690-bfc9-59332c86cc71 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:32.379963 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fc065-2945-497e-9cf1-b238a9acf252 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:32.522050 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28affbbc-97da-4690-bfc9-59332c86cc71 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:32.538053 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:32.566094 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc473f-59f3-4b09-8189-cdac4b2a088c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:32.567333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc473f-59f3-4b09-8189-cdac4b2a088c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:32.567819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc473f-59f3-4b09-8189-cdac4b2a088c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:32.568315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc473f-59f3-4b09-8189-cdac4b2a088c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:32.580390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc473f-59f3-4b09-8189-cdac4b2a088c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:32.588488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-71bc473f-59f3-4b09-8189-cdac4b2a088c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:29:32.588488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:32.588488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:32.588488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:32.588488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:32.588488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:32.588488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:32.588488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:32.588488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:32.588488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:32.588488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:32.588488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:32.588488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:32.588488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:32.588488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:32.588488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:32.589472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:32.589472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:32.589472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:32.589472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:32.589472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:32.589472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:32.589472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:32.589472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:32.589472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:32.589472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:32.589472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:32.589472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:32.589472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:32.589472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:32.589472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:32.589472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:32.590358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:32.590358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:32.590358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:32.590358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:32.590358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:32.590358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:32.590358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:32.590358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:32.590358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:32.590358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:32.590358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:29:32.590358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:32.593053 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 410/791] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:29:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 467 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:32.598992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc473f-59f3-4b09-8189-cdac4b2a088c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:32.618692 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670b225-0b21-46dd-a015-3d64ee972ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:32.619510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670b225-0b21-46dd-a015-3d64ee972ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:32.619798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670b225-0b21-46dd-a015-3d64ee972ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:32.620079 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670b225-0b21-46dd-a015-3d64ee972ce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:32.648441 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:32.655032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670b225-0b21-46dd-a015-3d64ee972ce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:32.656513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670b225-0b21-46dd-a015-3d64ee972ce5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:32.694295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-d670b225-0b21-46dd-a015-3d64ee972ce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:32.694295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:32.694295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:32.694295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:32.694295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:32.694295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:32.694295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:32.694295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:32.694295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:32.694295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:32.694295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:32.694295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:32.694295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:32.694295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:32.694295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:32.695249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:32.695249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:32.695249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:32.695249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:32.695249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:32.695249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:32.695249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:32.695249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:32.695249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:32.695249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:32.695249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:32.695249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:32.695249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:32.695249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:32.695249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:32.695249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:32.695249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:32.696139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:32.696139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:32.696139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:32.696139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:32.696520 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 382/792] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:32.700949 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670b225-0b21-46dd-a015-3d64ee972ce5 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:32.715427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc473f-59f3-4b09-8189-cdac4b2a088c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:32.751941 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5196c00-3aac-4ef2-b604-39a8daf67882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:32.752973 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5196c00-3aac-4ef2-b604-39a8daf67882 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:32.753462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5196c00-3aac-4ef2-b604-39a8daf67882 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:32.754001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5196c00-3aac-4ef2-b604-39a8daf67882 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:32.754977 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5196c00-3aac-4ef2-b604-39a8daf67882 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:32.755485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5196c00-3aac-4ef2-b604-39a8daf67882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:32.757343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5196c00-3aac-4ef2-b604-39a8daf67882 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:32.766479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 411/793] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:29:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:32.772396 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12b7d2-d77e-4d29-8c70-8dada629d209 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:32.772989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12b7d2-d77e-4d29-8c70-8dada629d209 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:32.773438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12b7d2-d77e-4d29-8c70-8dada629d209 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:32.779426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 412/794] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:32.779426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12b7d2-d77e-4d29-8c70-8dada629d209 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:32.808768 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:32.863698 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670b225-0b21-46dd-a015-3d64ee972ce5 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:32.883549 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12b7d2-d77e-4d29-8c70-8dada629d209 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:32.904471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae9b41-e8bd-4805-a7ba-0f4d7d0b21c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:32.905583 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae9b41-e8bd-4805-a7ba-0f4d7d0b21c6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:32.906114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae9b41-e8bd-4805-a7ba-0f4d7d0b21c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:32.906680 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae9b41-e8bd-4805-a7ba-0f4d7d0b21c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:32.907951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae9b41-e8bd-4805-a7ba-0f4d7d0b21c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:32.908474 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae9b41-e8bd-4805-a7ba-0f4d7d0b21c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:32.910331 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae9b41-e8bd-4805-a7ba-0f4d7d0b21c6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:32.920441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3b4b1-2680-4ee7-a898-313e323abc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:32.921244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3b4b1-2680-4ee7-a898-313e323abc76 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:32.921523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3b4b1-2680-4ee7-a898-313e323abc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:32.921800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3b4b1-2680-4ee7-a898-313e323abc76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:32.926441 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 383/795] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:32.928086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe994e-763d-47a7-8672-b41722356cc4 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:32.928653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe994e-763d-47a7-8672-b41722356cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:32.929199 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe994e-763d-47a7-8672-b41722356cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:32.933049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3b4b1-2680-4ee7-a898-313e323abc76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:32.934660 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3b4b1-2680-4ee7-a898-313e323abc76 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:32.947846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-58fe994e-763d-47a7-8672-b41722356cc4 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:32.949450 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-58fe994e-763d-47a7-8672-b41722356cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:32.953300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-c5e3b4b1-2680-4ee7-a898-313e323abc76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:32.953300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:32.953300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:32.953300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:32.953300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:32.953300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:32.953300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:32.953300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:32.953300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:32.953300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:32.953300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:32.953300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:32.953300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:32.953300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:32.953300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:32.954198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:32.954198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:32.954198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:32.954198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:32.954198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:32.954198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:32.954198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:32.954198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:32.954198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:32.954198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:32.954198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:32.954198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:32.954198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:32.954198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:32.954198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:32.954198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:32.954198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:32.955516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:32.955516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:32.955516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:32.955516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:32.958280 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 413/796] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:32.959132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3b4b1-2680-4ee7-a898-313e323abc76 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:33.008047 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:33.038142 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 384/797] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:33.039556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe994e-763d-47a7-8672-b41722356cc4 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:33.054902 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:33.074065 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3b4b1-2680-4ee7-a898-313e323abc76 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:33.104614 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf90a25-6ec4-4856-b765-dc2548075c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:33.105726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf90a25-6ec4-4856-b765-dc2548075c49 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:33.105726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf90a25-6ec4-4856-b765-dc2548075c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:33.106043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf90a25-6ec4-4856-b765-dc2548075c49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:33.113647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf90a25-6ec4-4856-b765-dc2548075c49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:33.114909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf90a25-6ec4-4856-b765-dc2548075c49 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:33.129926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-5bf90a25-6ec4-4856-b765-dc2548075c49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:33.129926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:33.129926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:33.129926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:33.129926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:33.129926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:33.129926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:33.129926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:33.129926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:33.129926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:33.129926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:33.129926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:33.129926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:33.129926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:33.129926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:33.130906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:33.130906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:33.130906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:33.130906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:33.130906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:33.130906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:33.130906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:33.130906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:33.130906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:33.130906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:33.130906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:33.130906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:33.130906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:33.130906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:33.130906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:33.130906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:33.130906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:33.132435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:33.132435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:33.132435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:33.132435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:33.132435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:33.132435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:33.132435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:33.132435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:33.132435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 414/798] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:33.132435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf90a25-6ec4-4856-b765-dc2548075c49 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:33.175402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe994e-763d-47a7-8672-b41722356cc4 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:33.216880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec11799-a337-4050-a388-bab3bc12ca0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:33.217842 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec11799-a337-4050-a388-bab3bc12ca0d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:33.218157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec11799-a337-4050-a388-bab3bc12ca0d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:33.218524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec11799-a337-4050-a388-bab3bc12ca0d None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:33.219151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec11799-a337-4050-a388-bab3bc12ca0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:33.219457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec11799-a337-4050-a388-bab3bc12ca0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:33.220836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec11799-a337-4050-a388-bab3bc12ca0d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:33.237192 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 385/799] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:33.237192 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec11799-a337-4050-a388-bab3bc12ca0d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:33.242891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf90a25-6ec4-4856-b765-dc2548075c49 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:33.278628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a315056-65b0-4834-9984-6b6467758999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:33.279424 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a315056-65b0-4834-9984-6b6467758999 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:33.279707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a315056-65b0-4834-9984-6b6467758999 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:33.279983 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a315056-65b0-4834-9984-6b6467758999 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:33.282408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:33.291855 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a315056-65b0-4834-9984-6b6467758999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:33.299496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-6a315056-65b0-4834-9984-6b6467758999 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:29:33.299496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:33.299496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:33.299496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:33.299496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:33.299496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:33.299496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:33.299496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:33.299496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:33.299496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:33.299496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:33.299496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:33.299496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:33.299496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:33.299496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:33.299496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:33.300437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:33.300437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:33.300437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:33.300437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:33.300437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:33.300437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:33.300437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:33.300437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:33.300437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:33.300437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:33.300437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:33.300437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:33.300437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:33.300437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:33.300437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:33.300437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:33.301281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:33.301281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:33.301281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:33.301281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:33.301281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:33.301281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:33.301281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:33.301281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:33.301281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:33.301281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:33.301281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:29:33.301281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:33.301281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 415/800] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:29:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:33.302309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20caa8-baa5-485c-a134-6ea03b2618e0 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:33.302596 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20caa8-baa5-485c-a134-6ea03b2618e0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:33.302869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20caa8-baa5-485c-a134-6ea03b2618e0 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:33.304272 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 416/801] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:33.308695 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20caa8-baa5-485c-a134-6ea03b2618e0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:33.314251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:33.349232 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec11799-a337-4050-a388-bab3bc12ca0d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:33.385211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bac112-efb2-4aaa-99ac-63159c3ff8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:33.385877 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bac112-efb2-4aaa-99ac-63159c3ff8eb None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:33.386711 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bac112-efb2-4aaa-99ac-63159c3ff8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:33.386993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bac112-efb2-4aaa-99ac-63159c3ff8eb None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:33.387780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bac112-efb2-4aaa-99ac-63159c3ff8eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:33.388089 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bac112-efb2-4aaa-99ac-63159c3ff8eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:33.389520 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bac112-efb2-4aaa-99ac-63159c3ff8eb None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:33.403340 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 386/802] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:33.407998 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bac112-efb2-4aaa-99ac-63159c3ff8eb None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:33.420120 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20caa8-baa5-485c-a134-6ea03b2618e0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:33.460482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a4be5-8b8e-423b-8519-90b3884e38d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:33.461690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a4be5-8b8e-423b-8519-90b3884e38d0 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:33.462516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a4be5-8b8e-423b-8519-90b3884e38d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:33.462947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a4be5-8b8e-423b-8519-90b3884e38d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:33.472689 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a4be5-8b8e-423b-8519-90b3884e38d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:33.474840 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a4be5-8b8e-423b-8519-90b3884e38d0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:33.490365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-7c8a4be5-8b8e-423b-8519-90b3884e38d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:29:33.490365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:33.490365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:33.490365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:33.490365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:33.490365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:33.490365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:33.490365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:33.490365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:33.490365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:33.490365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:33.490365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:33.490365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:33.490365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:33.490365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:33.490365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:33.491553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:33.497633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:33.497633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:33.497633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:33.497633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:33.497633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:33.497633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:33.497633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:33.497633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:33.497633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:29:33.497633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:33.497633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 417/803] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jul 12 13:29:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:33.497633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fffe48-c1a3-4a5e-ab31-b204a34a722a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:33.497633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fffe48-c1a3-4a5e-ab31-b204a34a722a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:33.498380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fffe48-c1a3-4a5e-ab31-b204a34a722a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:33.513852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-71fffe48-c1a3-4a5e-ab31-b204a34a722a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:33.515101 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-71fffe48-c1a3-4a5e-ab31-b204a34a722a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:33.527707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bac112-efb2-4aaa-99ac-63159c3ff8eb None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:33.529308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:33.564659 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e69b6-d6bc-4e98-bc50-4e1dc34a9791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:33.565336 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e69b6-d6bc-4e98-bc50-4e1dc34a9791 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:33.565622 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e69b6-d6bc-4e98-bc50-4e1dc34a9791 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:33.565905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e69b6-d6bc-4e98-bc50-4e1dc34a9791 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:33.566684 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e69b6-d6bc-4e98-bc50-4e1dc34a9791 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:33.566996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e69b6-d6bc-4e98-bc50-4e1dc34a9791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:33.568373 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e69b6-d6bc-4e98-bc50-4e1dc34a9791 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:33.578145 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 387/804] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:29:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:33.579186 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e69b6-d6bc-4e98-bc50-4e1dc34a9791 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:33.587985 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:33.597786 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 418/805] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:33.598651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fffe48-c1a3-4a5e-ab31-b204a34a722a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:33.704131 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e69b6-d6bc-4e98-bc50-4e1dc34a9791 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:33.725368 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fffe48-c1a3-4a5e-ab31-b204a34a722a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:33.739109 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f71db-b2aa-4b2f-b660-61a2e6451745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:33.740009 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f71db-b2aa-4b2f-b660-61a2e6451745 None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:33.740296 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f71db-b2aa-4b2f-b660-61a2e6451745 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:33.740593 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f71db-b2aa-4b2f-b660-61a2e6451745 None admin] PATH_INFO: `/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/dedf564ba5744a1fa70bb78222d26d5b/roles/057d9335b3b1412ebfb6bfb27f02f96a` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:33.755830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f71db-b2aa-4b2f-b660-61a2e6451745 None admin] RBAC: Authorizing `identity:create_grant(project_id=36aa0326fd0545dface07a9ddc308ccd, user_id=dedf564ba5744a1fa70bb78222d26d5b, role_id=057d9335b3b1412ebfb6bfb27f02f96a)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:33.757287 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f71db-b2aa-4b2f-b660-61a2e6451745 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:33.763562 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08177449-4977-47d1-b7de-eba8d5c7d59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:33.764301 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08177449-4977-47d1-b7de-eba8d5c7d59a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:33.764561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08177449-4977-47d1-b7de-eba8d5c7d59a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:33.764841 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08177449-4977-47d1-b7de-eba8d5c7d59a None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:33.765683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08177449-4977-47d1-b7de-eba8d5c7d59a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:33.766054 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08177449-4977-47d1-b7de-eba8d5c7d59a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:33.767796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08177449-4977-47d1-b7de-eba8d5c7d59a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:33.781991 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 388/806] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:29:33 2022] PUT /identity/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/dedf564ba5744a1fa70bb78222d26d5b/roles/057d9335b3b1412ebfb6bfb27f02f96a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:33.781991 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f71db-b2aa-4b2f-b660-61a2e6451745 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:33.803366 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:33.803671 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 419/807] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jul 12 13:29:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:33.805343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08177449-4977-47d1-b7de-eba8d5c7d59a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:33.876518 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:33.927443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f71db-b2aa-4b2f-b660-61a2e6451745 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:33.942795 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08177449-4977-47d1-b7de-eba8d5c7d59a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:33.970880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce7e6e-3212-4054-bd89-ef04b0f13140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:33.971690 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce7e6e-3212-4054-bd89-ef04b0f13140 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:33.971969 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce7e6e-3212-4054-bd89-ef04b0f13140 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:33.972235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce7e6e-3212-4054-bd89-ef04b0f13140 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:33.985435 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce7e6e-3212-4054-bd89-ef04b0f13140 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:33.986310 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0fc51-ebdd-4d8e-a516-0e4ce016d808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:33.986971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce7e6e-3212-4054-bd89-ef04b0f13140 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:33.987156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0fc51-ebdd-4d8e-a516-0e4ce016d808 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:33.987394 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0fc51-ebdd-4d8e-a516-0e4ce016d808 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:33.987668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0fc51-ebdd-4d8e-a516-0e4ce016d808 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:33.988330 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0fc51-ebdd-4d8e-a516-0e4ce016d808 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:33.989702 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0fc51-ebdd-4d8e-a516-0e4ce016d808 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:34.006207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-87ce7e6e-3212-4054-bd89-ef04b0f13140 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:34.006207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:34.006207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:34.006207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:34.006207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:34.006207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:34.006207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:34.006207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:34.006207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:34.006207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:34.006207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:34.006207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:34.006207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:34.006207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:34.006207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:34.007334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:34.007334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:34.007334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:34.007334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:34.007334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:34.007334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:34.007334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:34.007334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:34.007334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:34.007334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:34.007334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:34.007334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:34.007334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:34.007334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:34.007334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:34.007334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:34.007334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:34.008335 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:34.008335 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:34.008335 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:34.008335 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:34.008730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-b8b0fc51-ebdd-4d8e-a516-0e4ce016d808 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:29:34.008730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:34.008730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:34.008730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:34.008730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:34.008730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:34.008730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:34.008730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:34.008730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:29:34.008730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:34.008730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:34.008730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:34.008730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:34.008730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:34.008730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:34.008730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:34.009802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:34.009802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:34.009802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:34.009802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:34.009802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:34.009802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:34.009802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:34.009802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:34.009802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:34.009802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:34.009802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:34.009802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:34.009802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:34.009802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:34.009802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:34.009802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:34.011528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:34.011528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:34.011528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:34.011528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:34.011528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:34.011528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:34.011528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:34.011528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:34.011528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:34.011528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:34.011528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:34.011528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:34.011528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:34.011528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:29:34.011528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:34.013417 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 389/808] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:34.013417 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce7e6e-3212-4054-bd89-ef04b0f13140 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:34.016294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 420/809] 10.222.0.52 () {56 vars in 1129 bytes} [Tue Jul 12 13:29:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:34.016294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976f9ad-e144-4bd6-a928-64edee022207 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:34.016294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976f9ad-e144-4bd6-a928-64edee022207 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:34.016294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976f9ad-e144-4bd6-a928-64edee022207 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:34.016294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 421/810] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:34.016917 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976f9ad-e144-4bd6-a928-64edee022207 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:34.083169 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:34.122166 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce7e6e-3212-4054-bd89-ef04b0f13140 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:34.123164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976f9ad-e144-4bd6-a928-64edee022207 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:34.140180 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:34.159388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48faa595-6f45-491e-8930-497a2b3b0587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:34.159619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346b5a3-d3da-45d8-a4f6-b7c193bce54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:34.160051 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48faa595-6f45-491e-8930-497a2b3b0587 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:34.160158 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346b5a3-d3da-45d8-a4f6-b7c193bce54a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:34.160334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48faa595-6f45-491e-8930-497a2b3b0587 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:34.160442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346b5a3-d3da-45d8-a4f6-b7c193bce54a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:34.160609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48faa595-6f45-491e-8930-497a2b3b0587 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:34.160727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346b5a3-d3da-45d8-a4f6-b7c193bce54a None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:34.161385 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346b5a3-d3da-45d8-a4f6-b7c193bce54a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:34.161811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346b5a3-d3da-45d8-a4f6-b7c193bce54a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:34.163235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346b5a3-d3da-45d8-a4f6-b7c193bce54a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:34.180332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 422/811] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:34.180332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346b5a3-d3da-45d8-a4f6-b7c193bce54a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:34.244037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48faa595-6f45-491e-8930-497a2b3b0587 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:34.246058 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48faa595-6f45-491e-8930-497a2b3b0587 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:34.282339 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346b5a3-d3da-45d8-a4f6-b7c193bce54a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:34.314392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:34.320991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca2632-88ca-4e2f-82c4-368f069441a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:34.321665 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca2632-88ca-4e2f-82c4-368f069441a9 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:34.321969 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca2632-88ca-4e2f-82c4-368f069441a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:34.322262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca2632-88ca-4e2f-82c4-368f069441a9 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:34.322975 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca2632-88ca-4e2f-82c4-368f069441a9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:34.323288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca2632-88ca-4e2f-82c4-368f069441a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:34.324712 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca2632-88ca-4e2f-82c4-368f069441a9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:34.333947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 423/812] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Jul 12 13:29:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:34.333947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c569af-9c5e-4b05-b18e-dc19419903d9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:34.333947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c569af-9c5e-4b05-b18e-dc19419903d9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:34.333947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c569af-9c5e-4b05-b18e-dc19419903d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:34.348575 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-97c569af-9c5e-4b05-b18e-dc19419903d9 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:34.349776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-97c569af-9c5e-4b05-b18e-dc19419903d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:34.363751 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 390/813] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:29:34.363751 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48faa595-6f45-491e-8930-497a2b3b0587 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:34.390052 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:34.424611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 424/814] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:34.425434 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c569af-9c5e-4b05-b18e-dc19419903d9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:34.475747 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48faa595-6f45-491e-8930-497a2b3b0587 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:34.514449 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0211a-08b0-478b-9ecc-53b8421a43b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:34.515273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0211a-08b0-478b-9ecc-53b8421a43b4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:34.515564 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0211a-08b0-478b-9ecc-53b8421a43b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:34.515866 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0211a-08b0-478b-9ecc-53b8421a43b4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:34.523940 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0211a-08b0-478b-9ecc-53b8421a43b4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:34.525514 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0211a-08b0-478b-9ecc-53b8421a43b4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:34.532958 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c569af-9c5e-4b05-b18e-dc19419903d9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:34.541572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-f6a0211a-08b0-478b-9ecc-53b8421a43b4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:29:34.541572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:34.541572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:34.541572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:34.541572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:34.541572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:34.541572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:34.541572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:34.541572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:34.541572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:34.541572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:34.541572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:34.541572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:34.541572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:34.541572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:34.541572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:34.543057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:34.543987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:34.543987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:34.543987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:34.543987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:34.543987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:34.543987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:34.543987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:34.543987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:34.543987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:29:34.543987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:34.543987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 391/815] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:29:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:34.543987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0211a-08b0-478b-9ecc-53b8421a43b4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:34.571685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe9db4-8352-41af-9e64-e76d3f4ec123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:34.572343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe9db4-8352-41af-9e64-e76d3f4ec123 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:34.572624 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe9db4-8352-41af-9e64-e76d3f4ec123 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:34.572897 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe9db4-8352-41af-9e64-e76d3f4ec123 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:34.574155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe9db4-8352-41af-9e64-e76d3f4ec123 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:34.574268 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:34.574453 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe9db4-8352-41af-9e64-e76d3f4ec123 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:34.575825 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe9db4-8352-41af-9e64-e76d3f4ec123 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:34.581914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 425/816] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Jul 12 13:29:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:34.582893 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe9db4-8352-41af-9e64-e76d3f4ec123 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:34.622250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:34.662564 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0211a-08b0-478b-9ecc-53b8421a43b4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:34.692578 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe9db4-8352-41af-9e64-e76d3f4ec123 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:34.699225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252af346-806a-4c60-b90e-807a27718c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:34.700010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252af346-806a-4c60-b90e-807a27718c6e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:34.700285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252af346-806a-4c60-b90e-807a27718c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:34.700552 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252af346-806a-4c60-b90e-807a27718c6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:34.711146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252af346-806a-4c60-b90e-807a27718c6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:34.712597 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252af346-806a-4c60-b90e-807a27718c6e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:34.731000 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafdb94-e4d0-4417-a7b1-1b71cc2d7fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:34.731273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-252af346-806a-4c60-b90e-807a27718c6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:34.731273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:34.731273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:34.731273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:34.731273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:34.731273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:34.731273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:34.731273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:34.731273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:34.731273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:34.731273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:34.731273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:34.731273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:34.731273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:34.731273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:34.732117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:34.732117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:34.732117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:34.732117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:34.732117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:34.732117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:34.732117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:34.732117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:34.732117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:34.732117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:34.732117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:34.732117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:34.732117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:34.732117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:34.732117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:34.732117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:34.732117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:34.733018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:34.733018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:34.733018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:34.733018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:34.733018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 392/817] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:34.733297 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252af346-806a-4c60-b90e-807a27718c6e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:34.734711 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafdb94-e4d0-4417-a7b1-1b71cc2d7fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:34.735166 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafdb94-e4d0-4417-a7b1-1b71cc2d7fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:34.735638 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafdb94-e4d0-4417-a7b1-1b71cc2d7fd9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:34.736537 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafdb94-e4d0-4417-a7b1-1b71cc2d7fd9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:34.738221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafdb94-e4d0-4417-a7b1-1b71cc2d7fd9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:34.751060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-fdafdb94-e4d0-4417-a7b1-1b71cc2d7fd9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:29:34.751060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:34.751060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:34.751060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:34.751060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:34.751060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:34.751060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:34.751060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:34.751060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:29:34.751060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:34.751060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:34.751060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:34.751060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:34.751060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:34.751060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:34.751060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:34.751060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:34.752306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:34.752306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:34.752306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:34.752306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:34.752306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:34.752306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:34.752306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:34.752306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:34.752306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:34.752306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:34.752306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:34.752306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:34.752306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:34.752306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:34.752306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:34.752306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:34.753148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:34.753148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:34.753148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:34.753148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:34.753148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:34.753148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:34.753148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:34.753148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:34.753148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:34.753148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:34.753148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:34.753148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:34.753148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:29:34.753148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:34.753148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 426/818] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:34 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:34.753939 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafdb94-e4d0-4417-a7b1-1b71cc2d7fd9 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:34.821925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:34.848261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252af346-806a-4c60-b90e-807a27718c6e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:34.866796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:34.871351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafdb94-e4d0-4417-a7b1-1b71cc2d7fd9 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:34.885372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f0f31-7a5b-4b52-bea9-78e852e18b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:34.886599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f0f31-7a5b-4b52-bea9-78e852e18b08 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:34.886890 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f0f31-7a5b-4b52-bea9-78e852e18b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:34.887169 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f0f31-7a5b-4b52-bea9-78e852e18b08 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:34.888033 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f0f31-7a5b-4b52-bea9-78e852e18b08 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:34.888337 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f0f31-7a5b-4b52-bea9-78e852e18b08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:34.889791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f0f31-7a5b-4b52-bea9-78e852e18b08 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:34.908657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 393/819] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Jul 12 13:29:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:34.909375 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f0f31-7a5b-4b52-bea9-78e852e18b08 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:34.916424 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fac71-5a68-4088-8090-fb22d4766342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:34.917095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fac71-5a68-4088-8090-fb22d4766342 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:34.917419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fac71-5a68-4088-8090-fb22d4766342 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:34.917696 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fac71-5a68-4088-8090-fb22d4766342 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:34.918487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fac71-5a68-4088-8090-fb22d4766342 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:34.919871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fac71-5a68-4088-8090-fb22d4766342 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:34.944449 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 427/820] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jul 12 13:29:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:34.945521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fac71-5a68-4088-8090-fb22d4766342 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:35.060155 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f0f31-7a5b-4b52-bea9-78e852e18b08 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:35.108445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fac71-5a68-4088-8090-fb22d4766342 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:35.109146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:35.113779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909ee6e-cce3-4000-9f12-975e532d0f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:35.114479 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909ee6e-cce3-4000-9f12-975e532d0f5a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:35.114764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909ee6e-cce3-4000-9f12-975e532d0f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:35.115877 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909ee6e-cce3-4000-9f12-975e532d0f5a None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:35.115877 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5909ee6e-cce3-4000-9f12-975e532d0f5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:35.116063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5909ee6e-cce3-4000-9f12-975e532d0f5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:35.117411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5909ee6e-cce3-4000-9f12-975e532d0f5a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:35.130623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 394/821] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:35.131271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909ee6e-cce3-4000-9f12-975e532d0f5a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:35.156923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b0e9c-0cb2-40ac-9f7e-2585f6607b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:35.158234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b0e9c-0cb2-40ac-9f7e-2585f6607b99 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:35.158656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b0e9c-0cb2-40ac-9f7e-2585f6607b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:35.159059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b0e9c-0cb2-40ac-9f7e-2585f6607b99 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:35.159907 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b0e9c-0cb2-40ac-9f7e-2585f6607b99 None admin] RBAC: Adding query filter params (name=network) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:35.160369 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b0e9c-0cb2-40ac-9f7e-2585f6607b99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:35.161750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b0e9c-0cb2-40ac-9f7e-2585f6607b99 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:35.170358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 428/822] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jul 12 13:29:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:35.171526 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b0e9c-0cb2-40ac-9f7e-2585f6607b99 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:35.192351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:35.266871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909ee6e-cce3-4000-9f12-975e532d0f5a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:35.284921 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b0e9c-0cb2-40ac-9f7e-2585f6607b99 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:35.311535 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9130e-cf7c-4dec-936c-912a3e9d8261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:35.312946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9130e-cf7c-4dec-936c-912a3e9d8261 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:35.313859 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9130e-cf7c-4dec-936c-912a3e9d8261 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:35.314365 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9130e-cf7c-4dec-936c-912a3e9d8261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:35.335572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed9130e-cf7c-4dec-936c-912a3e9d8261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:35.337366 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed9130e-cf7c-4dec-936c-912a3e9d8261 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:35.338766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb486e1-fa6f-4e86-8bf9-3a67261b0f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:35.339603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb486e1-fa6f-4e86-8bf9-3a67261b0f1f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:35.339890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb486e1-fa6f-4e86-8bf9-3a67261b0f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:35.340171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb486e1-fa6f-4e86-8bf9-3a67261b0f1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:35.349274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb486e1-fa6f-4e86-8bf9-3a67261b0f1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:35.351615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb486e1-fa6f-4e86-8bf9-3a67261b0f1f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:35.359181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-9ed9130e-cf7c-4dec-936c-912a3e9d8261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:35.359181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:35.359181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:35.359181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:35.359181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:35.359181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:35.359181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:35.359181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:35.359181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:35.359181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:35.359181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:35.359181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:35.359181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:35.359181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:35.359181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:35.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:35.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:35.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:35.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:35.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:35.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:35.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:35.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:35.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:35.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:35.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:35.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:35.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:35.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:35.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:35.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:35.360328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:35.361228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:35.361228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:35.361228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:35.361228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:35.368508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 395/823] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:35.371213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9130e-cf7c-4dec-936c-912a3e9d8261 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:35.371770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-0cb486e1-fa6f-4e86-8bf9-3a67261b0f1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:35.371770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:35.371770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:35.371770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:35.371770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:35.371770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:35.371770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:35.371770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:35.371770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:35.371770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:35.371770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:35.371770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:35.371770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:35.371770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:35.371770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:35.372632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:35.372632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:35.372632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:35.372632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:35.372632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:35.372632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:35.372632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:35.372632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:35.372632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:35.372632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:35.372632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:35.372632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:35.372632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:35.372632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:35.372632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:35.372632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:35.372632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:35.373947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:35.373947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:35.373947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:35.373947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:35.373947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:35.373947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:35.373947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:35.373947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:35.373947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 429/824] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:35.374890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb486e1-fa6f-4e86-8bf9-3a67261b0f1f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:35.440014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:35.456381 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:35.484674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb486e1-fa6f-4e86-8bf9-3a67261b0f1f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:35.519456 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9130e-cf7c-4dec-936c-912a3e9d8261 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:35.525479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45d43b-c5ef-48e4-9776-40f95d6a81e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:35.526194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45d43b-c5ef-48e4-9776-40f95d6a81e2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:35.526480 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45d43b-c5ef-48e4-9776-40f95d6a81e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:35.526805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45d43b-c5ef-48e4-9776-40f95d6a81e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:35.527648 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45d43b-c5ef-48e4-9776-40f95d6a81e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:35.527964 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45d43b-c5ef-48e4-9776-40f95d6a81e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:35.529584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45d43b-c5ef-48e4-9776-40f95d6a81e2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:35.543385 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 430/825] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:35.544288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45d43b-c5ef-48e4-9776-40f95d6a81e2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:35.556091 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99061fd-2a4b-4c30-832b-51121fe37f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:35.556806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99061fd-2a4b-4c30-832b-51121fe37f6a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:35.557094 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99061fd-2a4b-4c30-832b-51121fe37f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:35.557368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99061fd-2a4b-4c30-832b-51121fe37f6a None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:35.558776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99061fd-2a4b-4c30-832b-51121fe37f6a None admin] RBAC: Adding query filter params (type=network) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:35.559085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99061fd-2a4b-4c30-832b-51121fe37f6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:35.560463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99061fd-2a4b-4c30-832b-51121fe37f6a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:35.565785 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 396/826] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jul 12 13:29:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:35.566685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99061fd-2a4b-4c30-832b-51121fe37f6a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:35.667866 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45d43b-c5ef-48e4-9776-40f95d6a81e2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:35.681548 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99061fd-2a4b-4c30-832b-51121fe37f6a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:35.691552 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:35.704907 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e8ee8-1f2d-419e-8acb-c06dc07b7003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:35.705579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e8ee8-1f2d-419e-8acb-c06dc07b7003 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:35.705858 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e8ee8-1f2d-419e-8acb-c06dc07b7003 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:35.706172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e8ee8-1f2d-419e-8acb-c06dc07b7003 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:35.706958 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e8ee8-1f2d-419e-8acb-c06dc07b7003 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:35.707268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e8ee8-1f2d-419e-8acb-c06dc07b7003 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:35.709185 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e8ee8-1f2d-419e-8acb-c06dc07b7003 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:35.710941 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:35.723760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e899dcd-f1e7-4d7d-91e6-dfc1199e240b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:35.724294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 431/827] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:35.724632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e899dcd-f1e7-4d7d-91e6-dfc1199e240b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:35.724977 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e899dcd-f1e7-4d7d-91e6-dfc1199e240b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:35.725168 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e8ee8-1f2d-419e-8acb-c06dc07b7003 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:35.725266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e899dcd-f1e7-4d7d-91e6-dfc1199e240b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:35.733442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e899dcd-f1e7-4d7d-91e6-dfc1199e240b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:35.734974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e899dcd-f1e7-4d7d-91e6-dfc1199e240b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:35.750099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-1e899dcd-f1e7-4d7d-91e6-dfc1199e240b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:35.750099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:35.750099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:35.750099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:35.750099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:35.750099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:35.750099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:35.750099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:35.750099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:35.750099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:35.750099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:35.750099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:35.750099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:35.750099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:35.750099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:35.751024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:35.751024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:35.751024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:35.751024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:35.751024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:35.751024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:35.751024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:35.751024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:35.751024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:35.751024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:35.751024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:35.751024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:35.751024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:35.751024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:35.751024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:35.751024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:35.751024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:35.752163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:35.752163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:35.752163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:35.752163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:35.752163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:35.752163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:35.752163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:35.752163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:35.752163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 397/828] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:35.753108 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e899dcd-f1e7-4d7d-91e6-dfc1199e240b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:35.831304 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e8ee8-1f2d-419e-8acb-c06dc07b7003 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:35.864742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e899dcd-f1e7-4d7d-91e6-dfc1199e240b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:35.867676 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae157c7-b305-4978-b5c2-7af4c696fcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:35.868343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae157c7-b305-4978-b5c2-7af4c696fcaf None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:35.868625 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae157c7-b305-4978-b5c2-7af4c696fcaf None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:35.868900 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae157c7-b305-4978-b5c2-7af4c696fcaf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:35.869641 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae157c7-b305-4978-b5c2-7af4c696fcaf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:35.871097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae157c7-b305-4978-b5c2-7af4c696fcaf None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:35.895523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 432/829] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jul 12 13:29:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:35.896570 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae157c7-b305-4978-b5c2-7af4c696fcaf None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:35.959501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:36.031051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae157c7-b305-4978-b5c2-7af4c696fcaf None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:36.037454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1416baf-a6b9-4ff5-9d2c-60a675357c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:36.038181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1416baf-a6b9-4ff5-9d2c-60a675357c8d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:36.038463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1416baf-a6b9-4ff5-9d2c-60a675357c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:36.038789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1416baf-a6b9-4ff5-9d2c-60a675357c8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:36.039637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1416baf-a6b9-4ff5-9d2c-60a675357c8d None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=0d3915cde0d04bcfb9b5ce8c89af1e5b) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:36.039946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1416baf-a6b9-4ff5-9d2c-60a675357c8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:36.041425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1416baf-a6b9-4ff5-9d2c-60a675357c8d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:36.048782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 398/830] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:35 2022] GET /identity/v3/role_assignments?user.id=0d3915cde0d04bcfb9b5ce8c89af1e5b&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=864a6966630544e8bd68bcafba905c6f => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:36.050288 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4754a-6317-47e6-b99d-863b81758f61 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:36.050573 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4754a-6317-47e6-b99d-863b81758f61 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:36.050843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4754a-6317-47e6-b99d-863b81758f61 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:36.052387 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 399/831] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:36.059575 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0482f-9c0b-4a6a-aa53-13f9dfc9ca11 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:36.059921 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0482f-9c0b-4a6a-aa53-13f9dfc9ca11 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:36.060110 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0482f-9c0b-4a6a-aa53-13f9dfc9ca11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:36.071604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293735f-66cb-4e3e-8a41-d7cf1a789c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:36.072314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293735f-66cb-4e3e-8a41-d7cf1a789c6b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:36.072596 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293735f-66cb-4e3e-8a41-d7cf1a789c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:36.072874 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293735f-66cb-4e3e-8a41-d7cf1a789c6b None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:36.074299 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293735f-66cb-4e3e-8a41-d7cf1a789c6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:36.074614 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293735f-66cb-4e3e-8a41-d7cf1a789c6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:36.076128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293735f-66cb-4e3e-8a41-d7cf1a789c6b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:36.091052 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 433/832] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:36.098467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293735f-66cb-4e3e-8a41-d7cf1a789c6b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:36.102094 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f1d0482f-9c0b-4a6a-aa53-13f9dfc9ca11 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:36.103357 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f1d0482f-9c0b-4a6a-aa53-13f9dfc9ca11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:36.150640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:36.206793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 400/833] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:29:36.233415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d0482f-9c0b-4a6a-aa53-13f9dfc9ca11 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:36.242971 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293735f-66cb-4e3e-8a41-d7cf1a789c6b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:36.251955 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:36.305742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a1f3b-3288-4f55-886a-2f0668f8f50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:36.307001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a1f3b-3288-4f55-886a-2f0668f8f50e None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:36.307283 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a1f3b-3288-4f55-886a-2f0668f8f50e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:36.307685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a1f3b-3288-4f55-886a-2f0668f8f50e None admin] PATH_INFO: `/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/937bacab8c1c45c0876b8e61f6f3b859/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:36.310814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a1f3b-3288-4f55-886a-2f0668f8f50e None admin] RBAC: Authorizing `identity:create_grant(project_id=36aa0326fd0545dface07a9ddc308ccd, user_id=937bacab8c1c45c0876b8e61f6f3b859, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:36.312313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a1f3b-3288-4f55-886a-2f0668f8f50e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:36.331404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 434/834] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:29:36 2022] PUT /identity/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/937bacab8c1c45c0876b8e61f6f3b859/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:36.381497 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d0482f-9c0b-4a6a-aa53-13f9dfc9ca11 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:36.423534 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72368e62-08a0-4d30-be29-526fac7460f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:36.424788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72368e62-08a0-4d30-be29-526fac7460f2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:36.425274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72368e62-08a0-4d30-be29-526fac7460f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:36.425775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72368e62-08a0-4d30-be29-526fac7460f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:36.438637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72368e62-08a0-4d30-be29-526fac7460f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:36.448843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-72368e62-08a0-4d30-be29-526fac7460f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:36.448843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:36.448843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:36.448843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:36.448843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:36.448843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:36.448843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:36.448843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:36.448843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:36.448843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:36.448843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:36.448843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:36.448843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:36.448843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:36.448843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:36.448843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:36.449858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:36.449858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:36.449858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:36.449858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:36.449858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:36.449858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:36.449858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:36.449858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:36.449858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:36.449858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:36.449858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:36.449858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:36.449858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:36.449858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:36.449858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:36.449858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:36.450779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:36.450779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:36.450779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:36.450779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:36.450779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:36.450779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:36.450779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:36.450779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:36.450779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:36.450779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:36.450779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:29:36.450779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:36.453776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 401/835] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:29:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:36.463344 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a1f3b-3288-4f55-886a-2f0668f8f50e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:36.573229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a1f3b-3288-4f55-886a-2f0668f8f50e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:36.609129 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4acfe1-b86a-44e8-8f44-4542614fa1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:36.609806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4acfe1-b86a-44e8-8f44-4542614fa1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:36.610123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4acfe1-b86a-44e8-8f44-4542614fa1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:36.610411 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4acfe1-b86a-44e8-8f44-4542614fa1d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:36.611161 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4acfe1-b86a-44e8-8f44-4542614fa1d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:36.611471 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4acfe1-b86a-44e8-8f44-4542614fa1d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:36.612865 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4acfe1-b86a-44e8-8f44-4542614fa1d5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:36.626558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 435/836] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:29:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:36.628135 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72368e62-08a0-4d30-be29-526fac7460f2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:36.654835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:36.739133 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72368e62-08a0-4d30-be29-526fac7460f2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:36.775331 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4fa13-f406-4470-807d-1df26c0f80ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:36.776136 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4fa13-f406-4470-807d-1df26c0f80ad None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:36.776426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4fa13-f406-4470-807d-1df26c0f80ad None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:36.777255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4fa13-f406-4470-807d-1df26c0f80ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:36.788801 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4fa13-f406-4470-807d-1df26c0f80ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:36.790321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4fa13-f406-4470-807d-1df26c0f80ad None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:36.808461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-fff4fa13-f406-4470-807d-1df26c0f80ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:36.808461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:36.808461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:36.808461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:36.808461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:36.808461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:36.808461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:36.808461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:36.808461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:36.808461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:36.808461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:36.808461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:36.808461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:36.808461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:36.808461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:36.810602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:36.810602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:36.810602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:36.810602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:36.810602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:36.810602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:36.810602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:36.810602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:36.810602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:36.810602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:36.810602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:36.810602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:36.810602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:36.810602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:36.810602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:36.810602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:36.810602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:36.812729 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:36.812729 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:36.812729 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:36.812729 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:36.812729 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 402/837] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:36.815739 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4acfe1-b86a-44e8-8f44-4542614fa1d5 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:36.858000 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:36.925392 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4acfe1-b86a-44e8-8f44-4542614fa1d5 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:36.961193 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f825be-da25-4c8f-831c-a2edf417754c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:36.961860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f825be-da25-4c8f-831c-a2edf417754c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:36.962230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f825be-da25-4c8f-831c-a2edf417754c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:36.962515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f825be-da25-4c8f-831c-a2edf417754c None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:36.963241 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f825be-da25-4c8f-831c-a2edf417754c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:36.963550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f825be-da25-4c8f-831c-a2edf417754c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:36.965484 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f825be-da25-4c8f-831c-a2edf417754c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:36.977389 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 436/838] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:36.983052 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4fa13-f406-4470-807d-1df26c0f80ad None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:37.062353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:37.088324 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4fa13-f406-4470-807d-1df26c0f80ad None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:37.123647 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a8abc-7180-491f-be56-e7387b6ba740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:37.124476 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a8abc-7180-491f-be56-e7387b6ba740 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:37.124762 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a8abc-7180-491f-be56-e7387b6ba740 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:37.125041 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a8abc-7180-491f-be56-e7387b6ba740 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:37.133428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a8abc-7180-491f-be56-e7387b6ba740 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:37.134908 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a8abc-7180-491f-be56-e7387b6ba740 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:37.150937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-232a8abc-7180-491f-be56-e7387b6ba740 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:29:37.150937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:37.150937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:37.150937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:37.150937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:37.150937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:37.150937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:37.150937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:37.150937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:37.150937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:37.150937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:37.150937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:37.150937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:37.150937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:37.150937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:37.150937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:37.152442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:37.154430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:37.154430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:37.154430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:37.154430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:37.154430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:37.154430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:37.154430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:37.154430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:37.154430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:29:37.154430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:37.154430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 403/839] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jul 12 13:29:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:37.158697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f825be-da25-4c8f-831c-a2edf417754c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:37.249047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:37.253802 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1b9b9-fb96-4b92-afb7-71263eb92bd8 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:37.254170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1b9b9-fb96-4b92-afb7-71263eb92bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:37.254453 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1b9b9-fb96-4b92-afb7-71263eb92bd8 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:37.256488 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 404/840] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:37.263430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203b7d9-f5cf-45c7-b0a3-c32535f965a5 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:37.263719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203b7d9-f5cf-45c7-b0a3-c32535f965a5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:37.264000 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203b7d9-f5cf-45c7-b0a3-c32535f965a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:37.267259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f825be-da25-4c8f-831c-a2edf417754c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:37.281461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4203b7d9-f5cf-45c7-b0a3-c32535f965a5 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:37.282748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4203b7d9-f5cf-45c7-b0a3-c32535f965a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:37.301558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63712a0-53c8-41ae-a973-1f59d2d73c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:37.302262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63712a0-53c8-41ae-a973-1f59d2d73c7a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:37.302544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63712a0-53c8-41ae-a973-1f59d2d73c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:37.302818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63712a0-53c8-41ae-a973-1f59d2d73c7a None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:37.303651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63712a0-53c8-41ae-a973-1f59d2d73c7a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:37.303997 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63712a0-53c8-41ae-a973-1f59d2d73c7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:37.305398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63712a0-53c8-41ae-a973-1f59d2d73c7a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:37.324562 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 437/841] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Jul 12 13:29:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:37.329812 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63712a0-53c8-41ae-a973-1f59d2d73c7a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:37.370777 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 405/842] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:37.387144 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203b7d9-f5cf-45c7-b0a3-c32535f965a5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:37.436318 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63712a0-53c8-41ae-a973-1f59d2d73c7a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:37.456866 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:37.472804 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8005f-ef5b-44d8-884c-ed0378315ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:37.473461 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8005f-ef5b-44d8-884c-ed0378315ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:37.473744 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8005f-ef5b-44d8-884c-ed0378315ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:37.474043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8005f-ef5b-44d8-884c-ed0378315ad1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:37.490973 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:37.505897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203b7d9-f5cf-45c7-b0a3-c32535f965a5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:37.544646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e8679-af32-4bf1-97a5-50b23613fc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:37.545810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e8679-af32-4bf1-97a5-50b23613fc7e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:37.546375 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e8679-af32-4bf1-97a5-50b23613fc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:37.546850 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e8679-af32-4bf1-97a5-50b23613fc7e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:37.547851 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e8679-af32-4bf1-97a5-50b23613fc7e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:37.549566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e8679-af32-4bf1-97a5-50b23613fc7e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:37.550939 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8005f-ef5b-44d8-884c-ed0378315ad1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:37.552533 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8005f-ef5b-44d8-884c-ed0378315ad1 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:37.564180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-e19e8679-af32-4bf1-97a5-50b23613fc7e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:29:37.564180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:37.564180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:37.564180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:37.564180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:37.564180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:37.564180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:37.564180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:37.564180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:29:37.564180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:37.564180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:37.564180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:37.564180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:37.564180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:37.564180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:37.564180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:37.564180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:37.565203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:37.565203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:37.565203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:37.565203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:37.565203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:37.565203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:37.565203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:37.565203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:37.565203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:37.565203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:37.565203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:37.565203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:37.565203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:37.565203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:37.565203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:37.565203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:37.566069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:37.566069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:37.566069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:37.566069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:37.566069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:37.566069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:37.566069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:37.566069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:37.566069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:37.566069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:37.566069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:37.566069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:37.566069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:29:37.566069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:37.573678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 406/843] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:29:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:37.574995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e8679-af32-4bf1-97a5-50b23613fc7e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:37.662599 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 438/844] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:37 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:29:37.663286 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8005f-ef5b-44d8-884c-ed0378315ad1 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:37.695532 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e8679-af32-4bf1-97a5-50b23613fc7e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:37.723189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:37.727203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:37.738141 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd525ba-87f8-4827-9414-d0027bd9b911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:37.738973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd525ba-87f8-4827-9414-d0027bd9b911 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:37.739259 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd525ba-87f8-4827-9414-d0027bd9b911 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:37.739534 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd525ba-87f8-4827-9414-d0027bd9b911 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:37.740573 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd525ba-87f8-4827-9414-d0027bd9b911 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:37.740890 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd525ba-87f8-4827-9414-d0027bd9b911 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:37.742538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd525ba-87f8-4827-9414-d0027bd9b911 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:37.753989 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 407/845] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:29:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:37.755783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd525ba-87f8-4827-9414-d0027bd9b911 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:37.788928 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8005f-ef5b-44d8-884c-ed0378315ad1 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:37.825130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef117a-e679-49b4-81a2-f013c2147126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:37.825973 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef117a-e679-49b4-81a2-f013c2147126 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:37.826234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef117a-e679-49b4-81a2-f013c2147126 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:37.826507 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef117a-e679-49b4-81a2-f013c2147126 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:37.838285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef117a-e679-49b4-81a2-f013c2147126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:37.839783 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef117a-e679-49b4-81a2-f013c2147126 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:37.858691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-ebef117a-e679-49b4-81a2-f013c2147126 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:37.858691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:37.858691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:37.858691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:37.858691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:37.858691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:37.858691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:37.858691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:37.858691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:37.858691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:37.858691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:37.858691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:37.858691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:37.858691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:37.858691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:37.859645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:37.859645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:37.859645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:37.859645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:37.859645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:37.859645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:37.859645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:37.859645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:37.859645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:37.859645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:37.859645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:37.859645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:37.859645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:37.859645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:37.859645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:37.859645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:37.859645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:37.860558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:37.860558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:37.860558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:37.860558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:37.864642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 439/846] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:37.865482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef117a-e679-49b4-81a2-f013c2147126 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:37.874261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd525ba-87f8-4827-9414-d0027bd9b911 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:37.918518 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f1d3f-1fd2-4998-b42b-0af749c94303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:37.919106 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f1d3f-1fd2-4998-b42b-0af749c94303 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:37.919389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f1d3f-1fd2-4998-b42b-0af749c94303 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:37.919663 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f1d3f-1fd2-4998-b42b-0af749c94303 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:37.920365 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f1d3f-1fd2-4998-b42b-0af749c94303 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:37.920685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f1d3f-1fd2-4998-b42b-0af749c94303 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:37.922538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f1d3f-1fd2-4998-b42b-0af749c94303 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:37.933987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 408/847] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:29:37 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:37.990381 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:37.994763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef117a-e679-49b4-81a2-f013c2147126 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:38.035740 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafc1be-ce03-4ebc-83cf-0e88833d1133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:38.036422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafc1be-ce03-4ebc-83cf-0e88833d1133 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:38.036707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafc1be-ce03-4ebc-83cf-0e88833d1133 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:38.036981 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafc1be-ce03-4ebc-83cf-0e88833d1133 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:38.037718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafc1be-ce03-4ebc-83cf-0e88833d1133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:38.038052 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafc1be-ce03-4ebc-83cf-0e88833d1133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:38.039463 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafc1be-ce03-4ebc-83cf-0e88833d1133 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:38.053265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 440/848] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:38.059076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f1d3f-1fd2-4998-b42b-0af749c94303 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:38.123892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:38.173518 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f1d3f-1fd2-4998-b42b-0af749c94303 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:38.207654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ee5c9-f256-4583-b29e-407cea880c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:38.208453 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ee5c9-f256-4583-b29e-407cea880c1d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:38.208737 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ee5c9-f256-4583-b29e-407cea880c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:38.209012 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ee5c9-f256-4583-b29e-407cea880c1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:38.216706 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ee5c9-f256-4583-b29e-407cea880c1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:38.218167 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ee5c9-f256-4583-b29e-407cea880c1d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:38.233162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-508ee5c9-f256-4583-b29e-407cea880c1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:38.233162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:38.233162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:38.233162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:38.233162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:38.233162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:38.233162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:38.233162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:38.233162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:38.233162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:38.233162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:38.233162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:38.233162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:38.233162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:38.233162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:38.234130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:38.234130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:38.234130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:38.234130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:38.234130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:38.234130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:38.234130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:38.234130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:38.234130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:38.234130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:38.234130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:38.234130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:38.234130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:38.234130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:38.234130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:38.234130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:38.234130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:38.235064 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:38.235064 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:38.235064 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:38.235064 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:38.235064 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:38.235064 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:38.235064 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:38.235064 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:38.235064 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 409/849] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:38.240505 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafc1be-ce03-4ebc-83cf-0e88833d1133 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:38.344278 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b8570-04a8-45bc-91cf-29ac48000fa9 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:38.344642 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b8570-04a8-45bc-91cf-29ac48000fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:38.344826 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b8570-04a8-45bc-91cf-29ac48000fa9 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:38.347112 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 410/850] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:38.348452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafc1be-ce03-4ebc-83cf-0e88833d1133 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:38.355342 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cfcf8-7566-4d61-b7d1-dac8be4e1eee None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:38.355515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cfcf8-7566-4d61-b7d1-dac8be4e1eee None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:38.355799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cfcf8-7566-4d61-b7d1-dac8be4e1eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:38.373405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-010cfcf8-7566-4d61-b7d1-dac8be4e1eee None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:38.374750 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-010cfcf8-7566-4d61-b7d1-dac8be4e1eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:38.389452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51353b-1291-4e7f-950a-df03fa3d7ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:38.390511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51353b-1291-4e7f-950a-df03fa3d7ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:38.390990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51353b-1291-4e7f-950a-df03fa3d7ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:38.391270 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51353b-1291-4e7f-950a-df03fa3d7ab4 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:38.392059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51353b-1291-4e7f-950a-df03fa3d7ab4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:38.392374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51353b-1291-4e7f-950a-df03fa3d7ab4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:38.393831 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51353b-1291-4e7f-950a-df03fa3d7ab4 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:38.407047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 441/851] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:38.412740 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51353b-1291-4e7f-950a-df03fa3d7ab4 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:38.436766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:38.452799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 411/852] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:29:38.454662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b4c1c-9ded-4ca9-a7e5-ff336288bda3 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:38.455191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b4c1c-9ded-4ca9-a7e5-ff336288bda3 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:38.455674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b4c1c-9ded-4ca9-a7e5-ff336288bda3 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:38.457520 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 412/853] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:38.468203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756a544-6573-4c6e-af7a-fbb355f118dd None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:38.468489 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756a544-6573-4c6e-af7a-fbb355f118dd None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:38.468764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756a544-6573-4c6e-af7a-fbb355f118dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:38.487707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7756a544-6573-4c6e-af7a-fbb355f118dd None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:38.488892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7756a544-6573-4c6e-af7a-fbb355f118dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:38.522490 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51353b-1291-4e7f-950a-df03fa3d7ab4 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:38.537880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:38.561425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278c73f-d970-4c32-900b-04da3fc40d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:38.562201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278c73f-d970-4c32-900b-04da3fc40d3b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:38.562611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278c73f-d970-4c32-900b-04da3fc40d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:38.562905 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278c73f-d970-4c32-900b-04da3fc40d3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:38.564003 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278c73f-d970-4c32-900b-04da3fc40d3b None admin] RBAC: Adding query filter params (role.id=057d9335b3b1412ebfb6bfb27f02f96a, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=dedf564ba5744a1fa70bb78222d26d5b) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:38.564385 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278c73f-d970-4c32-900b-04da3fc40d3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:38.566208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278c73f-d970-4c32-900b-04da3fc40d3b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:38.574024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 413/854] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:38.574024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756a544-6573-4c6e-af7a-fbb355f118dd None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:38.574585 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 442/855] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:38 2022] GET /identity/v3/role_assignments?user.id=dedf564ba5744a1fa70bb78222d26d5b&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=057d9335b3b1412ebfb6bfb27f02f96a => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:38.602694 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278c73f-d970-4c32-900b-04da3fc40d3b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:38.716547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756a544-6573-4c6e-af7a-fbb355f118dd None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:38.724470 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:38.748994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278c73f-d970-4c32-900b-04da3fc40d3b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:38.761822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f88ccd-b917-43a5-928b-f70f7e40f42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:38.762671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f88ccd-b917-43a5-928b-f70f7e40f42b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:38.762937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f88ccd-b917-43a5-928b-f70f7e40f42b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:38.763205 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f88ccd-b917-43a5-928b-f70f7e40f42b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:38.763875 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f88ccd-b917-43a5-928b-f70f7e40f42b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:38.765298 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f88ccd-b917-43a5-928b-f70f7e40f42b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:38.782783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-74f88ccd-b917-43a5-928b-f70f7e40f42b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:29:38.782783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:38.782783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:38.782783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:38.782783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:38.782783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:38.782783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:38.782783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:38.782783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:29:38.782783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:38.782783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:38.782783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:38.782783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:38.782783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:38.782783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:38.782783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:38.783775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:38.783775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:38.783775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:38.783775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:38.783775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:38.783775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:38.783775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:38.783775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:38.783775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:38.783775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:38.783775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:38.783775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:38.783775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:38.783775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:38.783775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:38.783775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:38.784643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:38.784643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:38.784643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:38.784643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:38.784643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:38.784643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:38.784643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:38.784643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:38.784643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:38.784643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:38.784643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:38.784643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:38.784643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:38.784643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:29:38.784643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:38.784643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 414/856] 10.222.0.52 () {56 vars in 1132 bytes} [Tue Jul 12 13:29:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:38.791555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f88ccd-b917-43a5-928b-f70f7e40f42b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:38.798832 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e37ca-1dee-4ce3-8e62-c3657e3cf8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:38.800420 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e37ca-1dee-4ce3-8e62-c3657e3cf8de None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:38.800915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e37ca-1dee-4ce3-8e62-c3657e3cf8de None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:38.801393 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e37ca-1dee-4ce3-8e62-c3657e3cf8de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:38.819829 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e37ca-1dee-4ce3-8e62-c3657e3cf8de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:38.829025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-567e37ca-1dee-4ce3-8e62-c3657e3cf8de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:29:38.829025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:38.829025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:38.829025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:38.829025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:38.829025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:38.829025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:38.829025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:38.829025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:38.829025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:38.829025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:38.829025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:38.829025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:38.829025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:38.829025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:38.829025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:38.830023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:38.830023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:38.830023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:38.830023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:38.830023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:38.830023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:38.830023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:38.830023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:38.830023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:38.830023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:38.830023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:38.830023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:38.830023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:38.830023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:38.830023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:38.830023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:38.830909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:38.830909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:38.830909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:38.830909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:38.830909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:38.830909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:38.830909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:38.830909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:38.830909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:38.830909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:38.830909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:29:38.830909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:38.830909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 443/857] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:29:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:38.838778 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e37ca-1dee-4ce3-8e62-c3657e3cf8de None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:38.848545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:38.919503 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f88ccd-b917-43a5-928b-f70f7e40f42b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:38.961206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c795df1-3cbb-4440-b7ae-1ab35c9ac406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:38.961437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e37ca-1dee-4ce3-8e62-c3657e3cf8de None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:38.962418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c795df1-3cbb-4440-b7ae-1ab35c9ac406 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:38.962933 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c795df1-3cbb-4440-b7ae-1ab35c9ac406 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:38.963426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c795df1-3cbb-4440-b7ae-1ab35c9ac406 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:38.964373 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c795df1-3cbb-4440-b7ae-1ab35c9ac406 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:38.964887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c795df1-3cbb-4440-b7ae-1ab35c9ac406 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:38.966513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c795df1-3cbb-4440-b7ae-1ab35c9ac406 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:38.972547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 415/858] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jul 12 13:29:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:38.978741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c795df1-3cbb-4440-b7ae-1ab35c9ac406 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:39.018143 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0a91e-65c8-407c-95ba-4f3372cc8f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:39.019234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0a91e-65c8-407c-95ba-4f3372cc8f46 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:39.020240 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0a91e-65c8-407c-95ba-4f3372cc8f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:39.021080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0a91e-65c8-407c-95ba-4f3372cc8f46 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:39.022040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0a91e-65c8-407c-95ba-4f3372cc8f46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:39.022464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0a91e-65c8-407c-95ba-4f3372cc8f46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:39.023869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0a91e-65c8-407c-95ba-4f3372cc8f46 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:39.033918 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 444/859] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:29:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:39.039999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0a91e-65c8-407c-95ba-4f3372cc8f46 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:39.045385 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:39.105182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c795df1-3cbb-4440-b7ae-1ab35c9ac406 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:39.108878 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:39.154301 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed649f4-7a1a-4fe7-b441-e62824df9199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:39.155279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed649f4-7a1a-4fe7-b441-e62824df9199 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:39.155567 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed649f4-7a1a-4fe7-b441-e62824df9199 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:39.156102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed649f4-7a1a-4fe7-b441-e62824df9199 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:39.157260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed649f4-7a1a-4fe7-b441-e62824df9199 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:39.157852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed649f4-7a1a-4fe7-b441-e62824df9199 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:39.159976 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed649f4-7a1a-4fe7-b441-e62824df9199 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:39.162554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0a91e-65c8-407c-95ba-4f3372cc8f46 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:39.167942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 416/860] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jul 12 13:29:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:39.207608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9200e-f1b3-4dc5-98bb-fc620663fa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:39.208436 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9200e-f1b3-4dc5-98bb-fc620663fa95 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:39.208722 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9200e-f1b3-4dc5-98bb-fc620663fa95 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:39.208999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9200e-f1b3-4dc5-98bb-fc620663fa95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:39.221859 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9200e-f1b3-4dc5-98bb-fc620663fa95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:39.223787 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9200e-f1b3-4dc5-98bb-fc620663fa95 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:39.243775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-cad9200e-f1b3-4dc5-98bb-fc620663fa95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:39.243775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:39.243775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:39.243775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:39.243775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:39.243775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:39.243775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:39.243775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:39.243775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:39.243775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:39.243775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:39.243775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:39.243775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:39.243775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:39.243775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:39.244763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:39.244763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:39.244763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:39.244763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:39.244763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:39.244763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:39.244763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:39.244763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:39.244763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:39.244763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:39.244763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:39.244763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:39.244763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:39.244763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:39.244763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:39.244763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:39.244763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:39.246285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:39.246285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:39.246285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:39.246285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:39.246285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 445/861] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:39.255132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed649f4-7a1a-4fe7-b441-e62824df9199 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:39.364578 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed649f4-7a1a-4fe7-b441-e62824df9199 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:39.399599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b3926-c41c-4e50-9ed4-e9e603408981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:39.400271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b3926-c41c-4e50-9ed4-e9e603408981 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:39.400557 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b3926-c41c-4e50-9ed4-e9e603408981 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:39.400833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b3926-c41c-4e50-9ed4-e9e603408981 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:39.402106 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b3926-c41c-4e50-9ed4-e9e603408981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:39.402418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b3926-c41c-4e50-9ed4-e9e603408981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:39.403790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b3926-c41c-4e50-9ed4-e9e603408981 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:39.416269 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 417/862] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:39.422176 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9200e-f1b3-4dc5-98bb-fc620663fa95 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:39.486063 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:39.533132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9200e-f1b3-4dc5-98bb-fc620663fa95 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:39.569634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac480f58-75ed-4023-bbe8-468cf9e3ad92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:39.570474 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac480f58-75ed-4023-bbe8-468cf9e3ad92 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:39.570759 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac480f58-75ed-4023-bbe8-468cf9e3ad92 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:39.571036 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac480f58-75ed-4023-bbe8-468cf9e3ad92 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:39.578933 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac480f58-75ed-4023-bbe8-468cf9e3ad92 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:39.580396 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac480f58-75ed-4023-bbe8-468cf9e3ad92 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:39.595586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-ac480f58-75ed-4023-bbe8-468cf9e3ad92 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:29:39.595586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:39.595586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:39.595586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:39.595586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:39.595586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:39.595586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:39.595586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:39.595586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:39.595586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:39.595586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:39.595586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:39.595586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:39.595586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:39.595586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:39.595586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:39.596514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:39.597447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:39.597447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:39.597447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:39.597447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:39.597447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:39.597447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:39.597447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:39.597447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:39.597447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:29:39.597447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:39.597447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 446/863] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:29:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:39.602499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b3926-c41c-4e50-9ed4-e9e603408981 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:39.636689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:39.711923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b3926-c41c-4e50-9ed4-e9e603408981 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:39.749125 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd3b18-b9f6-43eb-8208-330bab5197b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:39.749713 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd3b18-b9f6-43eb-8208-330bab5197b4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:39.750018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd3b18-b9f6-43eb-8208-330bab5197b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:39.750298 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd3b18-b9f6-43eb-8208-330bab5197b4 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:39.751376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd3b18-b9f6-43eb-8208-330bab5197b4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:39.751695 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd3b18-b9f6-43eb-8208-330bab5197b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:39.753126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd3b18-b9f6-43eb-8208-330bab5197b4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:39.771921 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 418/864] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Jul 12 13:29:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:39.777673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac480f58-75ed-4023-bbe8-468cf9e3ad92 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:39.883554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac480f58-75ed-4023-bbe8-468cf9e3ad92 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:39.902790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:39.922102 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6feeef5-e7f3-4bda-9d60-539c12467d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:39.922805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6feeef5-e7f3-4bda-9d60-539c12467d13 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:39.923092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6feeef5-e7f3-4bda-9d60-539c12467d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:39.923423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6feeef5-e7f3-4bda-9d60-539c12467d13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:39.994701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6feeef5-e7f3-4bda-9d60-539c12467d13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:39.996224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6feeef5-e7f3-4bda-9d60-539c12467d13 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:40.099108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 447/865] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:39 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:29:40.107067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd3b18-b9f6-43eb-8208-330bab5197b4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:40.162306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25571cf-c9ff-42da-b5e0-4697d01774c4 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:40.163166 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25571cf-c9ff-42da-b5e0-4697d01774c4 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:40.163652 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25571cf-c9ff-42da-b5e0-4697d01774c4 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:40.165528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 448/866] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:29:40.171282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852ad00-0c13-4646-8229-56a0155e964b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:40.172047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852ad00-0c13-4646-8229-56a0155e964b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:40.172834 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852ad00-0c13-4646-8229-56a0155e964b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:40.195135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1852ad00-0c13-4646-8229-56a0155e964b None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:40.197424 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1852ad00-0c13-4646-8229-56a0155e964b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:40.199636 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:40.218287 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd3b18-b9f6-43eb-8208-330bab5197b4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:40.218493 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:40.256341 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8ae84-acfa-4f8f-bc4f-fcbf186099d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:40.257188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8ae84-acfa-4f8f-bc4f-fcbf186099d2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:40.257477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8ae84-acfa-4f8f-bc4f-fcbf186099d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:40.257753 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8ae84-acfa-4f8f-bc4f-fcbf186099d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:40.268908 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8ae84-acfa-4f8f-bc4f-fcbf186099d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:40.270459 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8ae84-acfa-4f8f-bc4f-fcbf186099d2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:40.289464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-72c8ae84-acfa-4f8f-bc4f-fcbf186099d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:40.289464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:40.289464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:40.289464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:40.289464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:40.289464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:40.289464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:40.289464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:40.289464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:40.289464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:40.289464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:40.289464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:40.289464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:40.289464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:40.289464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:40.290634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:40.290634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:40.290634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:40.290634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:40.290634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:40.290634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:40.290634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:40.290634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:40.290634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:40.290634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:40.290634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:40.290634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:40.290634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:40.290634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:40.290634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:40.290634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:40.290634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:40.291538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:40.291538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:40.291538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:40.291538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:40.291538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 419/867] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:40.293665 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 449/868] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:40.301712 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8ae84-acfa-4f8f-bc4f-fcbf186099d2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:40.316553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852ad00-0c13-4646-8229-56a0155e964b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:40.416820 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8ae84-acfa-4f8f-bc4f-fcbf186099d2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:40.447501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852ad00-0c13-4646-8229-56a0155e964b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:40.454160 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b5bc7-113a-4d94-b775-9250b58cdbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:40.455070 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b5bc7-113a-4d94-b775-9250b58cdbd1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:40.455571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b5bc7-113a-4d94-b775-9250b58cdbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:40.456037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b5bc7-113a-4d94-b775-9250b58cdbd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:40.456980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b5bc7-113a-4d94-b775-9250b58cdbd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:40.457478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b5bc7-113a-4d94-b775-9250b58cdbd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:40.459071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b5bc7-113a-4d94-b775-9250b58cdbd1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:40.466121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:40.478291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 420/869] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:40.479524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b5bc7-113a-4d94-b775-9250b58cdbd1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:40.489213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57c6d5-1e61-41b6-8a5e-12758a2f7e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:40.490346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57c6d5-1e61-41b6-8a5e-12758a2f7e34 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:40.490643 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57c6d5-1e61-41b6-8a5e-12758a2f7e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:40.490924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57c6d5-1e61-41b6-8a5e-12758a2f7e34 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:40.491666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57c6d5-1e61-41b6-8a5e-12758a2f7e34 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:40.493429 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57c6d5-1e61-41b6-8a5e-12758a2f7e34 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:40.518060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 450/870] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jul 12 13:29:40 2022] POST /identity/v3/services => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:29:40.608955 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b5bc7-113a-4d94-b775-9250b58cdbd1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:40.653478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c7b28-4ca1-463e-9182-f58c105cfa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:40.654314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c7b28-4ca1-463e-9182-f58c105cfa30 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:40.654892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c7b28-4ca1-463e-9182-f58c105cfa30 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:40.655176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c7b28-4ca1-463e-9182-f58c105cfa30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:40.663215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c7b28-4ca1-463e-9182-f58c105cfa30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:40.664705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c7b28-4ca1-463e-9182-f58c105cfa30 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:40.681323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-620c7b28-4ca1-463e-9182-f58c105cfa30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:40.681323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:40.681323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:40.681323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:40.681323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:40.681323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:40.681323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:40.681323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:40.681323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:40.681323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:40.681323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:40.681323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:40.681323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:40.681323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:40.681323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:40.682310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:40.682310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:40.682310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:40.682310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:40.682310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:40.682310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:40.682310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:40.682310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:40.682310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:40.682310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:40.682310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:40.682310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:40.682310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:40.682310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:40.682310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:40.682310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:40.682310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:40.683793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:40.683793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:40.683793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:40.683793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:40.683793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:40.683793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:40.683793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:40.683793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:40.683793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 421/871] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:40.691407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57c6d5-1e61-41b6-8a5e-12758a2f7e34 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:40.702471 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:40.806752 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57c6d5-1e61-41b6-8a5e-12758a2f7e34 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:40.840609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a5d06-023e-4885-9e28-e941a12f24d3 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:40.840923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a5d06-023e-4885-9e28-e941a12f24d3 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:40.841258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a5d06-023e-4885-9e28-e941a12f24d3 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:40.842931 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 422/872] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:29:40.844885 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91d1ff-6857-4c76-a851-ce27604cba25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:40.846606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91d1ff-6857-4c76-a851-ce27604cba25 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:40.846606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91d1ff-6857-4c76-a851-ce27604cba25 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:40.846606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91d1ff-6857-4c76-a851-ce27604cba25 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:40.849559 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91d1ff-6857-4c76-a851-ce27604cba25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:40.849559 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91d1ff-6857-4c76-a851-ce27604cba25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:40.849559 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91d1ff-6857-4c76-a851-ce27604cba25 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:40.853210 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb966972-de91-445a-a556-4ae3bba1ee92 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:40.853210 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb966972-de91-445a-a556-4ae3bba1ee92 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:40.853210 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb966972-de91-445a-a556-4ae3bba1ee92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:40.863470 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 451/873] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:40.869616 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91d1ff-6857-4c76-a851-ce27604cba25 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:40.874992 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-fb966972-de91-445a-a556-4ae3bba1ee92 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:40.876669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fb966972-de91-445a-a556-4ae3bba1ee92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:40.938443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:40.956353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 423/874] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:40 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jul 12 13:29:40.957134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:40.976101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb966972-de91-445a-a556-4ae3bba1ee92 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:40.994936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91d1ff-6857-4c76-a851-ce27604cba25 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:41.032675 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721553e1-0d2d-4d55-ac6a-3e01fce1ddc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:41.033486 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721553e1-0d2d-4d55-ac6a-3e01fce1ddc3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:41.033746 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721553e1-0d2d-4d55-ac6a-3e01fce1ddc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:41.034066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721553e1-0d2d-4d55-ac6a-3e01fce1ddc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:41.035245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721553e1-0d2d-4d55-ac6a-3e01fce1ddc3 None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=937bacab8c1c45c0876b8e61f6f3b859) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:41.035359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721553e1-0d2d-4d55-ac6a-3e01fce1ddc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:41.036736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721553e1-0d2d-4d55-ac6a-3e01fce1ddc3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:41.049988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 452/875] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:40 2022] GET /identity/v3/role_assignments?user.id=937bacab8c1c45c0876b8e61f6f3b859&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=864a6966630544e8bd68bcafba905c6f => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:41.103137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb966972-de91-445a-a556-4ae3bba1ee92 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:41.141599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4201f-6a2c-4c73-843c-a1ead691e89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:41.142457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4201f-6a2c-4c73-843c-a1ead691e89a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:41.142744 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4201f-6a2c-4c73-843c-a1ead691e89a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:41.143017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4201f-6a2c-4c73-843c-a1ead691e89a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:41.155713 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4201f-6a2c-4c73-843c-a1ead691e89a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:41.164392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-7dd4201f-6a2c-4c73-843c-a1ead691e89a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:29:41.164392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:41.164392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:41.164392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:41.164392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:41.164392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:41.164392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:41.164392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:41.164392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:41.164392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:41.164392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:41.164392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:41.164392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:41.164392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:41.164392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:41.164392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:41.165406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:41.165406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:41.165406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:41.165406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:41.165406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:41.165406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:41.165406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:41.165406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:41.165406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:41.165406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:41.165406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:41.165406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:41.165406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:41.165406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:41.165406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:41.165406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:41.166318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:41.166318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:41.166318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:41.166318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:41.166318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:41.166318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:41.166318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:41.166318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:41.166318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:41.166318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:41.166318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:29:41.166318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:41.170094 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 424/876] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Jul 12 13:29:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:41.172854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721553e1-0d2d-4d55-ac6a-3e01fce1ddc3 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:41.286236 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721553e1-0d2d-4d55-ac6a-3e01fce1ddc3 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:41.297597 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a828b-a6df-4939-b4bf-0185b3b31495 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:41.298202 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a828b-a6df-4939-b4bf-0185b3b31495 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:41.298202 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a828b-a6df-4939-b4bf-0185b3b31495 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:41.299824 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 425/877] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:41.305621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d933e-684c-45f8-ae8a-5d9cd01998f6 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:41.305915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d933e-684c-45f8-ae8a-5d9cd01998f6 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:41.306232 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d933e-684c-45f8-ae8a-5d9cd01998f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:41.326612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f90d933e-684c-45f8-ae8a-5d9cd01998f6 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:41.328055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f90d933e-684c-45f8-ae8a-5d9cd01998f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:41.328775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:41.338533 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85413537-2965-449b-ad12-f4944157c615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:41.339770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85413537-2965-449b-ad12-f4944157c615 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:41.340059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85413537-2965-449b-ad12-f4944157c615 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:41.340341 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85413537-2965-449b-ad12-f4944157c615 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:41.341594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85413537-2965-449b-ad12-f4944157c615 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:41.341912 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85413537-2965-449b-ad12-f4944157c615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:41.344215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85413537-2965-449b-ad12-f4944157c615 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:41.356147 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 453/878] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Jul 12 13:29:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:41.361942 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85413537-2965-449b-ad12-f4944157c615 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:41.374872 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:41.407161 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 426/879] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:41 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:29:41.430067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d933e-684c-45f8-ae8a-5d9cd01998f6 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:41.497203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85413537-2965-449b-ad12-f4944157c615 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:41.545308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7545d-f1c7-4d5f-985f-6d56f04885fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:41.546063 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7545d-f1c7-4d5f-985f-6d56f04885fc None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:41.546357 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7545d-f1c7-4d5f-985f-6d56f04885fc None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:41.546636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7545d-f1c7-4d5f-985f-6d56f04885fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:41.560136 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d933e-684c-45f8-ae8a-5d9cd01998f6 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:41.561636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7545d-f1c7-4d5f-985f-6d56f04885fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:41.564645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7545d-f1c7-4d5f-985f-6d56f04885fc None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:41.589708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-c4b7545d-f1c7-4d5f-985f-6d56f04885fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:41.589708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:41.589708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:41.589708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:41.589708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:41.589708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:41.589708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:41.589708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:41.589708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:41.589708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:41.589708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:41.589708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:41.589708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:41.589708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:41.589708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:41.591049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:41.591049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:41.591049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:41.591049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:41.591049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:41.591049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:41.591049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:41.591049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:41.591049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:41.591049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:41.591049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:41.591049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:41.591049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:41.591049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:41.591049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:41.591049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:41.591049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:41.592068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:41.592068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:41.592068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:41.592068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:41.592068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 454/880] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:41.597181 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7545d-f1c7-4d5f-985f-6d56f04885fc None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:41.614490 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdb00f-e4cb-4de4-abf7-978e839bd6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:41.616022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdb00f-e4cb-4de4-abf7-978e839bd6b6 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:41.616796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdb00f-e4cb-4de4-abf7-978e839bd6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:41.617291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdb00f-e4cb-4de4-abf7-978e839bd6b6 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:41.619070 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdb00f-e4cb-4de4-abf7-978e839bd6b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:41.620526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdb00f-e4cb-4de4-abf7-978e839bd6b6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:41.658003 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 427/881] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jul 12 13:29:41 2022] POST /identity/v3/services => generated 269 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:29:41.664025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:41.728304 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7545d-f1c7-4d5f-985f-6d56f04885fc None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:41.775478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18a4e4-9c86-4317-93c0-21587ac2a073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:41.776158 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18a4e4-9c86-4317-93c0-21587ac2a073 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:41.776481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18a4e4-9c86-4317-93c0-21587ac2a073 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:41.776762 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18a4e4-9c86-4317-93c0-21587ac2a073 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:41.777564 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18a4e4-9c86-4317-93c0-21587ac2a073 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:41.777878 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18a4e4-9c86-4317-93c0-21587ac2a073 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:41.779397 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18a4e4-9c86-4317-93c0-21587ac2a073 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:41.798201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 455/882] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:41.799783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdb00f-e4cb-4de4-abf7-978e839bd6b6 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:41.838982 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:41.922452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdb00f-e4cb-4de4-abf7-978e839bd6b6 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:41.958306 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db439f-9879-44e9-8ec6-46638660a10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:41.959448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db439f-9879-44e9-8ec6-46638660a10c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:41.959917 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db439f-9879-44e9-8ec6-46638660a10c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:41.960363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db439f-9879-44e9-8ec6-46638660a10c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:41.968307 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db439f-9879-44e9-8ec6-46638660a10c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:41.970081 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db439f-9879-44e9-8ec6-46638660a10c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:41.985071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-76db439f-9879-44e9-8ec6-46638660a10c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:29:41.985071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:41.985071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:41.985071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:41.985071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:41.985071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:41.985071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:41.985071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:41.985071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:41.985071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:41.985071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:41.985071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:41.985071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:41.985071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:41.985071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:41.985071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:41.986026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:41.986979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:41.986979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:41.986979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:41.986979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:41.986979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:41.986979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:41.986979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:41.986979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:41.986979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:29:41.986979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:41.993527 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 428/883] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jul 12 13:29:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:41.995552 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18a4e4-9c86-4317-93c0-21587ac2a073 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:42.096762 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:42.100001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18a4e4-9c86-4317-93c0-21587ac2a073 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:42.137976 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae580fb-3ec8-4e36-967e-993550063f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:42.138658 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae580fb-3ec8-4e36-967e-993550063f58 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:42.138948 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae580fb-3ec8-4e36-967e-993550063f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:42.139228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae580fb-3ec8-4e36-967e-993550063f58 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:42.140072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae580fb-3ec8-4e36-967e-993550063f58 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:42.140387 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae580fb-3ec8-4e36-967e-993550063f58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:42.141843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae580fb-3ec8-4e36-967e-993550063f58 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:42.161998 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 456/884] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Jul 12 13:29:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:42.168240 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db439f-9879-44e9-8ec6-46638660a10c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:42.265030 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:42.271878 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db439f-9879-44e9-8ec6-46638660a10c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:42.309790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb94bd8-b80d-479a-b454-376ec290a57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:42.310489 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb94bd8-b80d-479a-b454-376ec290a57a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:42.310725 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb94bd8-b80d-479a-b454-376ec290a57a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:42.311000 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb94bd8-b80d-479a-b454-376ec290a57a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:42.379950 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb94bd8-b80d-479a-b454-376ec290a57a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:42.381404 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb94bd8-b80d-479a-b454-376ec290a57a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:42.475177 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:42.487726 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 429/885] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:42 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:29:42.489124 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae580fb-3ec8-4e36-967e-993550063f58 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:42.601892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae580fb-3ec8-4e36-967e-993550063f58 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:42.637770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de9352-0f34-41fe-83ce-4c7295131b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:42.638525 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de9352-0f34-41fe-83ce-4c7295131b53 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:42.638811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de9352-0f34-41fe-83ce-4c7295131b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:42.639091 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de9352-0f34-41fe-83ce-4c7295131b53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:42.649734 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de9352-0f34-41fe-83ce-4c7295131b53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:42.651763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de9352-0f34-41fe-83ce-4c7295131b53 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:42.671447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-23de9352-0f34-41fe-83ce-4c7295131b53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:42.671447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:42.671447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:42.671447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:42.671447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:42.671447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:42.671447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:42.671447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:42.671447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:42.671447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:42.671447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:42.671447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:42.671447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:42.671447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:42.671447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:42.672443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:42.672443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:42.672443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:42.672443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:42.672443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:42.672443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:42.672443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:42.672443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:42.672443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:42.672443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:42.672443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:42.672443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:42.672443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:42.672443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:42.672443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:42.672443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:42.672443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:42.673639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:42.673639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:42.673639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:42.673639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:42.673639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 457/886] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:42.679800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb94bd8-b80d-479a-b454-376ec290a57a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:42.755194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e890dd-af3c-4cc5-8321-99d10513adab None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:42.755523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e890dd-af3c-4cc5-8321-99d10513adab None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:42.755759 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e890dd-af3c-4cc5-8321-99d10513adab None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:42.757397 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 458/887] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:42.763428 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0d905-84ff-4690-ab4f-09a663c7bde9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:42.763719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0d905-84ff-4690-ab4f-09a663c7bde9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:42.764000 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0d905-84ff-4690-ab4f-09a663c7bde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:42.782398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3bd0d905-84ff-4690-ab4f-09a663c7bde9 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:42.783698 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3bd0d905-84ff-4690-ab4f-09a663c7bde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:42.786804 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:42.788176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb94bd8-b80d-479a-b454-376ec290a57a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:42.825710 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dea8b4-2e70-4a60-b78a-9a9ecf9f1457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:42.826749 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dea8b4-2e70-4a60-b78a-9a9ecf9f1457 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:42.827254 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dea8b4-2e70-4a60-b78a-9a9ecf9f1457 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:42.827748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dea8b4-2e70-4a60-b78a-9a9ecf9f1457 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:42.828660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dea8b4-2e70-4a60-b78a-9a9ecf9f1457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:42.829214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dea8b4-2e70-4a60-b78a-9a9ecf9f1457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:42.830918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dea8b4-2e70-4a60-b78a-9a9ecf9f1457 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:42.852441 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 430/888] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:42.854254 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dea8b4-2e70-4a60-b78a-9a9ecf9f1457 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:42.866054 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 459/889] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:42 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:29:42.881848 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0d905-84ff-4690-ab4f-09a663c7bde9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:42.917568 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:42.967068 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dea8b4-2e70-4a60-b78a-9a9ecf9f1457 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:42.996461 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0d905-84ff-4690-ab4f-09a663c7bde9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:43.003891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1767f-a309-4039-b0bc-9c7dbf4be442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:43.005094 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1767f-a309-4039-b0bc-9c7dbf4be442 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.005620 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1767f-a309-4039-b0bc-9c7dbf4be442 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.006168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1767f-a309-4039-b0bc-9c7dbf4be442 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:43.014272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c1767f-a309-4039-b0bc-9c7dbf4be442 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:43.016123 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c1767f-a309-4039-b0bc-9c7dbf4be442 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:43.032822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-d5c1767f-a309-4039-b0bc-9c7dbf4be442 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:43.032822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:43.032822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:43.032822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:43.032822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:43.032822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:43.032822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:43.032822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:43.032822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:43.032822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:43.032822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:43.032822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:43.032822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:43.032822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:43.032822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:43.033730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:43.033730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:43.033730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:43.033730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:43.033730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:43.033730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:43.033730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:43.033730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:43.033730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:43.033730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:43.033730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:43.033730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:43.033730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:43.033730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:43.033730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:43.033730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:43.033730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:43.034657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:43.034657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:43.034657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:43.034657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:43.034657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:43.034657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:43.034657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:43.034657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:43.035081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f7284-23fd-4621-b0f1-0d8935de6cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:43.035081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f7284-23fd-4621-b0f1-0d8935de6cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.035081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f7284-23fd-4621-b0f1-0d8935de6cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.035081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f7284-23fd-4621-b0f1-0d8935de6cc6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:43.035346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f7284-23fd-4621-b0f1-0d8935de6cc6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:43.036674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f7284-23fd-4621-b0f1-0d8935de6cc6 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:43.042379 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 431/890] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:43.043931 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1767f-a309-4039-b0bc-9c7dbf4be442 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:43.049290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-9a2f7284-23fd-4621-b0f1-0d8935de6cc6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:29:43.049290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:43.049290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:43.049290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:43.049290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:43.049290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:43.049290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:43.049290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:43.049290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:29:43.049290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:43.049290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:43.049290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:43.049290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:43.049290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:43.049290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:43.049290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:43.049290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:43.050293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:43.050293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:43.050293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:43.050293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:43.050293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:43.050293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:43.050293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:43.050293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:43.050293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:43.050293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:43.050293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:43.050293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:43.050293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:43.050293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:43.050293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:43.050293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:43.051529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:43.051529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:43.051529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:43.051529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:43.051529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:43.051529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:43.051529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:43.051529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:43.051529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:43.051529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:43.051529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:43.051529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:43.051529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:29:43.051529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:43.054035 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 460/891] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:29:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:43.056486 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f7284-23fd-4621-b0f1-0d8935de6cc6 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:43.060472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:43.156977 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1767f-a309-4039-b0bc-9c7dbf4be442 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:43.166798 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f7284-23fd-4621-b0f1-0d8935de6cc6 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:43.172742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:43.194841 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082e924-0b33-481f-bbb1-802a9fabbf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:43.195758 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082e924-0b33-481f-bbb1-802a9fabbf7e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.196139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082e924-0b33-481f-bbb1-802a9fabbf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.196510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082e924-0b33-481f-bbb1-802a9fabbf7e None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:43.197392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082e924-0b33-481f-bbb1-802a9fabbf7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:43.197802 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082e924-0b33-481f-bbb1-802a9fabbf7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:43.200146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082e924-0b33-481f-bbb1-802a9fabbf7e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:43.209531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4f1e9-853c-4744-ab4d-481ddff34e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:43.210651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4f1e9-853c-4744-ab4d-481ddff34e7f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.211090 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4f1e9-853c-4744-ab4d-481ddff34e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.211539 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4f1e9-853c-4744-ab4d-481ddff34e7f None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:43.212396 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4f1e9-853c-4744-ab4d-481ddff34e7f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:43.212861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4f1e9-853c-4744-ab4d-481ddff34e7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:43.214484 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4f1e9-853c-4744-ab4d-481ddff34e7f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:43.217268 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 432/892] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:43.218738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ddaf4-6887-4f1e-ada2-f87a56e88969 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.219032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ddaf4-6887-4f1e-ada2-f87a56e88969 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.219312 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ddaf4-6887-4f1e-ada2-f87a56e88969 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:43.220797 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 461/893] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:29:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:43.220908 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 433/894] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:43.221554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4f1e9-853c-4744-ab4d-481ddff34e7f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:43.226011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48e375-bbb6-40ec-b247-36d5f1921b58 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.226141 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48e375-bbb6-40ec-b247-36d5f1921b58 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.226348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48e375-bbb6-40ec-b247-36d5f1921b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:43.243086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8c48e375-bbb6-40ec-b247-36d5f1921b58 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:43.244274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8c48e375-bbb6-40ec-b247-36d5f1921b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:43.292611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:43.329980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 434/895] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:29:43.329980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c48e375-bbb6-40ec-b247-36d5f1921b58 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:43.330862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4f1e9-853c-4744-ab4d-481ddff34e7f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:43.364315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958a27e-7327-42a2-8e29-43e4c52bde64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:43.364966 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958a27e-7327-42a2-8e29-43e4c52bde64 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.365249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958a27e-7327-42a2-8e29-43e4c52bde64 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.365522 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958a27e-7327-42a2-8e29-43e4c52bde64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:43.366373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958a27e-7327-42a2-8e29-43e4c52bde64 None admin] RBAC: Adding query filter params (role.id=19deffe64f224b68880ce003b4af8b2b, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=dedf564ba5744a1fa70bb78222d26d5b) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:43.366683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958a27e-7327-42a2-8e29-43e4c52bde64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:43.368117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958a27e-7327-42a2-8e29-43e4c52bde64 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:43.386363 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 462/896] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:43 2022] GET /identity/v3/role_assignments?user.id=dedf564ba5744a1fa70bb78222d26d5b&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=19deffe64f224b68880ce003b4af8b2b => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:43.387444 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958a27e-7327-42a2-8e29-43e4c52bde64 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:43.441537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c48e375-bbb6-40ec-b247-36d5f1921b58 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:43.446336 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:43.488097 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482e972-3151-427b-8370-273ddc2729b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:43.489376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482e972-3151-427b-8370-273ddc2729b5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.489878 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482e972-3151-427b-8370-273ddc2729b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.490389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482e972-3151-427b-8370-273ddc2729b5 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:43.491327 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482e972-3151-427b-8370-273ddc2729b5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:43.491870 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482e972-3151-427b-8370-273ddc2729b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:43.493501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482e972-3151-427b-8370-273ddc2729b5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:43.500979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 435/897] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:29:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:43.506799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482e972-3151-427b-8370-273ddc2729b5 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:43.549477 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958a27e-7327-42a2-8e29-43e4c52bde64 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:43.588382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:43.594331 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71305913-0810-4cb5-bb90-5d38f06974b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:43.595882 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71305913-0810-4cb5-bb90-5d38f06974b7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.596603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71305913-0810-4cb5-bb90-5d38f06974b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.597061 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71305913-0810-4cb5-bb90-5d38f06974b7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:43.598077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71305913-0810-4cb5-bb90-5d38f06974b7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:43.600491 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71305913-0810-4cb5-bb90-5d38f06974b7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:43.616938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-71305913-0810-4cb5-bb90-5d38f06974b7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:29:43.616938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:43.616938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:43.616938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:43.616938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:43.616938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:43.616938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:43.616938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:43.616938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:29:43.616938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:43.616938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:43.616938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:43.616938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:43.616938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:43.616938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:43.616938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:43.616938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:43.618445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:43.618445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:43.618445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:43.618445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:43.618445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:43.618445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:43.618445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:43.618445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:43.618445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:43.618445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:43.618445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:43.618445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:43.618445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:43.618445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:43.618445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:43.618445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:43.619265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:43.619265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:43.619265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:43.619265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:43.619265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:43.619265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:43.619265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:43.619265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:43.619265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:43.619265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:43.619265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:43.619265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:43.619265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:29:43.619265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:43.628071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 463/898] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jul 12 13:29:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:43.628924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71305913-0810-4cb5-bb90-5d38f06974b7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:43.646526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482e972-3151-427b-8370-273ddc2729b5 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:43.693725 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbcf01-cb1f-4ede-ad83-dff4a7b969d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:43.695144 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbcf01-cb1f-4ede-ad83-dff4a7b969d5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.695638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbcf01-cb1f-4ede-ad83-dff4a7b969d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.696332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbcf01-cb1f-4ede-ad83-dff4a7b969d5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:43.697456 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fbcf01-cb1f-4ede-ad83-dff4a7b969d5 None admin] RBAC: Adding query filter params (interface=public, service_id=3482949df6c647ae97be2477afaca205, region_id=RegionOne) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:43.697949 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fbcf01-cb1f-4ede-ad83-dff4a7b969d5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:43.700095 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fbcf01-cb1f-4ede-ad83-dff4a7b969d5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:43.708539 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 436/899] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Jul 12 13:29:43 2022] GET /identity/v3/endpoints?service_id=3482949df6c647ae97be2477afaca205&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:43.713836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbcf01-cb1f-4ede-ad83-dff4a7b969d5 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:43.744595 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71305913-0810-4cb5-bb90-5d38f06974b7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:43.746541 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:43.789081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6bd706-d3ae-4284-8e46-aaea3da21342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:43.789753 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6bd706-d3ae-4284-8e46-aaea3da21342 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.790062 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6bd706-d3ae-4284-8e46-aaea3da21342 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.790337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6bd706-d3ae-4284-8e46-aaea3da21342 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:43.791062 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6bd706-d3ae-4284-8e46-aaea3da21342 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:43.791347 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6bd706-d3ae-4284-8e46-aaea3da21342 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:43.793377 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6bd706-d3ae-4284-8e46-aaea3da21342 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:43.798778 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 464/900] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jul 12 13:29:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:43.805307 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6bd706-d3ae-4284-8e46-aaea3da21342 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:43.830557 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbcf01-cb1f-4ede-ad83-dff4a7b969d5 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:43.876402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfffd91a-a208-4791-b0fb-d7a57d76b37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:43.877089 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfffd91a-a208-4791-b0fb-d7a57d76b37a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.877360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfffd91a-a208-4791-b0fb-d7a57d76b37a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.877630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfffd91a-a208-4791-b0fb-d7a57d76b37a None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:43.878346 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfffd91a-a208-4791-b0fb-d7a57d76b37a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:43.880569 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:43.881042 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfffd91a-a208-4791-b0fb-d7a57d76b37a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:43.887516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 437/901] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jul 12 13:29:43 2022] GET /identity/v3/services => generated 1268 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:43.888857 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e515e10-63f8-4ccd-946f-8a4097780741 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.889172 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e515e10-63f8-4ccd-946f-8a4097780741 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.889403 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e515e10-63f8-4ccd-946f-8a4097780741 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:43.894671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 438/902] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:43.897241 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5db8e0-4c35-448b-8ff3-0092ad0b19a1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.897522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5db8e0-4c35-448b-8ff3-0092ad0b19a1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.897794 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5db8e0-4c35-448b-8ff3-0092ad0b19a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:43.916166 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-db5db8e0-4c35-448b-8ff3-0092ad0b19a1 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:43.918360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-db5db8e0-4c35-448b-8ff3-0092ad0b19a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:43.943662 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6bd706-d3ae-4284-8e46-aaea3da21342 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:43.992304 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7db5dc-bb5c-45ba-b905-7acbc9746108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:43.993442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7db5dc-bb5c-45ba-b905-7acbc9746108 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.997998 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 439/903] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:43.998736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7db5dc-bb5c-45ba-b905-7acbc9746108 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.999149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7db5dc-bb5c-45ba-b905-7acbc9746108 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:44.000092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7db5dc-bb5c-45ba-b905-7acbc9746108 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:44.000534 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7db5dc-bb5c-45ba-b905-7acbc9746108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:44.002446 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7db5dc-bb5c-45ba-b905-7acbc9746108 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:44.008379 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 465/904] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jul 12 13:29:43 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:44.023223 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5db8e0-4c35-448b-8ff3-0092ad0b19a1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:44.064002 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:44.142593 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5db8e0-4c35-448b-8ff3-0092ad0b19a1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:44.181929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d68a7d-90c6-497a-a830-9a14a948d63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:44.183244 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d68a7d-90c6-497a-a830-9a14a948d63f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:44.183699 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d68a7d-90c6-497a-a830-9a14a948d63f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:44.184123 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d68a7d-90c6-497a-a830-9a14a948d63f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:44.185110 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d68a7d-90c6-497a-a830-9a14a948d63f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:44.186553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d68a7d-90c6-497a-a830-9a14a948d63f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:44.201153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-25d68a7d-90c6-497a-a830-9a14a948d63f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:29:44.201153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:44.201153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:44.201153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:44.201153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:44.201153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:44.201153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:44.201153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:44.201153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:29:44.201153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:44.201153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:44.201153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:44.201153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:44.201153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:44.201153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:44.201153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:44.202143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:44.202143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:44.202143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:44.202143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:44.202143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:44.202143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:44.202143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:44.202143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:44.202143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:44.202143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:44.202143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:44.202143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:44.202143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:44.202143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:44.202143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:44.202143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:44.202973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:44.202973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:44.202973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:44.202973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:44.202973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:44.202973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:44.202973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:44.202973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:44.202973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:44.202973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:44.202973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:44.202973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:44.202973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:44.202973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:29:44.202973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:44.202973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 440/905] 10.222.0.52 () {56 vars in 1132 bytes} [Tue Jul 12 13:29:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:44.208606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7db5dc-bb5c-45ba-b905-7acbc9746108 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:44.313110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7db5dc-bb5c-45ba-b905-7acbc9746108 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:44.337196 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:44.350038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f0bb3-b962-4219-b5b1-4008199470fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:44.350716 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f0bb3-b962-4219-b5b1-4008199470fe None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:44.350999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f0bb3-b962-4219-b5b1-4008199470fe None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:44.351275 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f0bb3-b962-4219-b5b1-4008199470fe None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:44.351987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f0bb3-b962-4219-b5b1-4008199470fe None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:44.352296 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f0bb3-b962-4219-b5b1-4008199470fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:44.353672 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f0bb3-b962-4219-b5b1-4008199470fe None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:44.359433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 466/906] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jul 12 13:29:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:44.364981 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d68a7d-90c6-497a-a830-9a14a948d63f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:44.462411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:44.474338 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d68a7d-90c6-497a-a830-9a14a948d63f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:44.514935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f5579-5b3f-4b3c-9f75-f72193b0ec95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:44.516000 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f5579-5b3f-4b3c-9f75-f72193b0ec95 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:44.516489 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f5579-5b3f-4b3c-9f75-f72193b0ec95 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:44.516991 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f5579-5b3f-4b3c-9f75-f72193b0ec95 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:44.517939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f5579-5b3f-4b3c-9f75-f72193b0ec95 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:44.518505 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f5579-5b3f-4b3c-9f75-f72193b0ec95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:44.520163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f5579-5b3f-4b3c-9f75-f72193b0ec95 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:44.527247 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 441/907] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jul 12 13:29:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:44.532880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f0bb3-b962-4219-b5b1-4008199470fe None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:44.639795 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f0bb3-b962-4219-b5b1-4008199470fe None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:44.675077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298179e-10ae-4ee0-9cc4-a264254bd8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:44.675748 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298179e-10ae-4ee0-9cc4-a264254bd8ef None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:44.676034 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298179e-10ae-4ee0-9cc4-a264254bd8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:44.676312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298179e-10ae-4ee0-9cc4-a264254bd8ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:44.677113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298179e-10ae-4ee0-9cc4-a264254bd8ef None admin] RBAC: Adding query filter params (interface=public, service_id=9c0549d6c8b0462caf3453e31e03883a, region_id=RegionOne) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:44.677429 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298179e-10ae-4ee0-9cc4-a264254bd8ef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:44.678935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298179e-10ae-4ee0-9cc4-a264254bd8ef None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:44.687839 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 467/908] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Jul 12 13:29:44 2022] GET /identity/v3/endpoints?service_id=9c0549d6c8b0462caf3453e31e03883a&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:44.693259 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f5579-5b3f-4b3c-9f75-f72193b0ec95 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:44.817298 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f5579-5b3f-4b3c-9f75-f72193b0ec95 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:44.851705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102d786-bed8-4e23-aac6-fd8e34094879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:44.852420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102d786-bed8-4e23-aac6-fd8e34094879 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:44.852707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102d786-bed8-4e23-aac6-fd8e34094879 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:44.852969 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102d786-bed8-4e23-aac6-fd8e34094879 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:44.853735 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102d786-bed8-4e23-aac6-fd8e34094879 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:44.855247 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102d786-bed8-4e23-aac6-fd8e34094879 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:44.861571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 442/909] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jul 12 13:29:44 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:45.470118 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea99107-be26-4787-b07f-47526e3364ca None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:45.470531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea99107-be26-4787-b07f-47526e3364ca None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:45.470725 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea99107-be26-4787-b07f-47526e3364ca None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:45.472320 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 468/910] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:45.477609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688358b-9fb7-4ba0-9212-01b6667681af None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:45.478116 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688358b-9fb7-4ba0-9212-01b6667681af None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:45.478203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688358b-9fb7-4ba0-9212-01b6667681af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:45.495921 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d688358b-9fb7-4ba0-9212-01b6667681af None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:45.497113 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d688358b-9fb7-4ba0-9212-01b6667681af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:45.510979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:45.566982 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 443/911] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:45.590260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea99107-be26-4787-b07f-47526e3364ca None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:45.646136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:45.698253 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea99107-be26-4787-b07f-47526e3364ca None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:45.734275 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d95e6-497d-4758-bbfe-1a18a8cbf7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:45.735059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d95e6-497d-4758-bbfe-1a18a8cbf7cc None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:45.735348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d95e6-497d-4758-bbfe-1a18a8cbf7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:45.735626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d95e6-497d-4758-bbfe-1a18a8cbf7cc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:45.746207 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d95e6-497d-4758-bbfe-1a18a8cbf7cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:45.754915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-a25d95e6-497d-4758-bbfe-1a18a8cbf7cc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:29:45.754915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:45.754915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:45.754915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:45.754915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:45.754915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:45.754915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:45.754915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:45.754915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:45.754915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:45.754915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:45.754915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:45.754915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:45.754915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:45.754915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:45.754915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:45.755890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:45.755890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:45.755890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:45.755890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:45.755890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:45.755890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:45.755890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:45.755890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:45.755890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:45.755890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:45.755890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:45.755890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:45.755890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:45.755890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:45.755890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:45.755890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:45.756776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:45.756776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:45.756776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:45.756776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:45.756776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:45.756776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:45.756776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:45.756776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:45.756776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:45.756776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:45.756776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:29:45.756776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:45.756776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 469/912] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Jul 12 13:29:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:45.762875 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688358b-9fb7-4ba0-9212-01b6667681af None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:45.863678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688358b-9fb7-4ba0-9212-01b6667681af None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:45.901893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae5935-399c-4f6a-ab61-00b05c011dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:45.902607 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae5935-399c-4f6a-ab61-00b05c011dcb None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:45.902865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae5935-399c-4f6a-ab61-00b05c011dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:45.903137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae5935-399c-4f6a-ab61-00b05c011dcb None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:45.903887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae5935-399c-4f6a-ab61-00b05c011dcb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:45.904190 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae5935-399c-4f6a-ab61-00b05c011dcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:45.905551 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae5935-399c-4f6a-ab61-00b05c011dcb None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:45.914499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 444/913] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Jul 12 13:29:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:45.923245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d95e6-497d-4758-bbfe-1a18a8cbf7cc None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:46.032075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d95e6-497d-4758-bbfe-1a18a8cbf7cc None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:46.045258 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:46.048787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad8f07-d061-4c34-bfe5-47a441cf5215 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:46.049079 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad8f07-d061-4c34-bfe5-47a441cf5215 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:46.049362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad8f07-d061-4c34-bfe5-47a441cf5215 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:46.054599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 445/914] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:46.056371 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779ab72-204b-4796-8934-d85a670ecfb5 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:46.056663 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779ab72-204b-4796-8934-d85a670ecfb5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:46.056939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779ab72-204b-4796-8934-d85a670ecfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:46.068233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab7b88-baa5-4075-a9b4-2c2f621ad8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:46.069029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab7b88-baa5-4075-a9b4-2c2f621ad8b1 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:46.069313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab7b88-baa5-4075-a9b4-2c2f621ad8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:46.069589 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab7b88-baa5-4075-a9b4-2c2f621ad8b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:46.077750 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d779ab72-204b-4796-8934-d85a670ecfb5 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:46.079621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d779ab72-204b-4796-8934-d85a670ecfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:46.083434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:46.084274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab7b88-baa5-4075-a9b4-2c2f621ad8b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:46.085940 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab7b88-baa5-4075-a9b4-2c2f621ad8b1 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:46.108249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-d9ab7b88-baa5-4075-a9b4-2c2f621ad8b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:46.108249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:46.108249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:46.108249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:46.108249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:46.108249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:46.108249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:46.108249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:46.108249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:46.108249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:46.108249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:46.108249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:46.108249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:46.108249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:46.108249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:46.109447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:46.109447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:46.109447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:46.109447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:46.109447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:46.109447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:46.109447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:46.109447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:46.109447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:46.109447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:46.109447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:46.109447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:46.109447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:46.109447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:46.109447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:46.109447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:46.109447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:46.110571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:46.110571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:46.110571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:46.110571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:46.110571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 470/915] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:46.116683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab7b88-baa5-4075-a9b4-2c2f621ad8b1 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:46.174052 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 446/916] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:46.196613 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779ab72-204b-4796-8934-d85a670ecfb5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:46.243261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab7b88-baa5-4075-a9b4-2c2f621ad8b1 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:46.283247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3458647-61d7-4796-bfc5-db67d896bdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:46.283542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3458647-61d7-4796-bfc5-db67d896bdb6 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:46.283825 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3458647-61d7-4796-bfc5-db67d896bdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:46.284100 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3458647-61d7-4796-bfc5-db67d896bdb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:46.284820 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3458647-61d7-4796-bfc5-db67d896bdb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:46.285129 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3458647-61d7-4796-bfc5-db67d896bdb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:46.286617 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3458647-61d7-4796-bfc5-db67d896bdb6 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:46.299089 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 471/917] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:46.300148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9341a9-a2e9-4d82-9e5f-62143b01634a None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:46.300433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9341a9-a2e9-4d82-9e5f-62143b01634a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:46.300707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9341a9-a2e9-4d82-9e5f-62143b01634a None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:46.302103 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 472/918] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:46.306600 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9341a9-a2e9-4d82-9e5f-62143b01634a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:46.320313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779ab72-204b-4796-8934-d85a670ecfb5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:46.333526 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:46.357681 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abcd460-1a67-4372-aca1-4734296c83c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:46.358516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcd460-1a67-4372-aca1-4734296c83c6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:46.358804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcd460-1a67-4372-aca1-4734296c83c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:46.359082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcd460-1a67-4372-aca1-4734296c83c6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:46.359744 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abcd460-1a67-4372-aca1-4734296c83c6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:46.361118 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abcd460-1a67-4372-aca1-4734296c83c6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:46.374207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-1abcd460-1a67-4372-aca1-4734296c83c6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:29:46.374207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:46.374207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:46.374207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:46.374207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:46.374207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:46.374207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:46.374207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:46.374207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:29:46.374207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:46.374207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:46.374207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:46.374207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:46.374207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:46.374207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:46.374207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:46.374207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:46.375258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:46.375258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:46.375258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:46.375258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:46.375258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:46.375258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:46.375258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:46.375258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:46.375258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:46.375258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:46.375258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:46.375258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:46.375258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:46.375258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:46.375258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:46.375258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:46.377096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:46.377096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:46.377096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:46.377096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:46.377096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:46.377096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:46.377096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:46.377096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:46.377096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:46.377096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:46.377096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:46.377096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:46.377096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:29:46.377096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:46.377096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 447/919] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:29:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:46.377945 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5da3d-5d64-458e-839a-3a258b4551aa None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:46.377945 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5da3d-5d64-458e-839a-3a258b4551aa None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:46.377945 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5da3d-5d64-458e-839a-3a258b4551aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:46.384158 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:46.395725 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1ea5da3d-5d64-458e-839a-3a258b4551aa None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:46.396973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1ea5da3d-5d64-458e-839a-3a258b4551aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:46.423198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9341a9-a2e9-4d82-9e5f-62143b01634a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:46.467747 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 448/920] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:46.468668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5da3d-5d64-458e-839a-3a258b4551aa None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:46.478186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e2c40-2f85-445f-b044-a01d08955606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:46.479144 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e2c40-2f85-445f-b044-a01d08955606 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:46.479496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e2c40-2f85-445f-b044-a01d08955606 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:46.479844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e2c40-2f85-445f-b044-a01d08955606 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:46.488053 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e2c40-2f85-445f-b044-a01d08955606 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:46.489594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e2c40-2f85-445f-b044-a01d08955606 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:46.506359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-c16e2c40-2f85-445f-b044-a01d08955606 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:29:46.506359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:46.506359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:46.506359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:46.506359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:46.506359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:46.506359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:46.506359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:46.506359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:46.506359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:46.506359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:46.506359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:46.506359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:46.506359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:46.506359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:46.506359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:46.507443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:46.508718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:46.508718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:46.508718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:46.508718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:46.508718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:46.508718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:46.508718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:46.508718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:46.508718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:29:46.508718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:46.508718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 473/921] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jul 12 13:29:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:46.508718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e2c40-2f85-445f-b044-a01d08955606 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:46.584951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5da3d-5d64-458e-839a-3a258b4551aa None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:46.597828 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:46.615087 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e2c40-2f85-445f-b044-a01d08955606 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:46.624969 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947beff-60d8-4ebd-8f68-6d260b4a2b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:46.625923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947beff-60d8-4ebd-8f68-6d260b4a2b27 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:46.626712 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947beff-60d8-4ebd-8f68-6d260b4a2b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:46.627141 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947beff-60d8-4ebd-8f68-6d260b4a2b27 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:46.628007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8947beff-60d8-4ebd-8f68-6d260b4a2b27 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:46.628518 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8947beff-60d8-4ebd-8f68-6d260b4a2b27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:46.630174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8947beff-60d8-4ebd-8f68-6d260b4a2b27 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:46.631527 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:46.637428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 449/922] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:29:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:46.640838 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947beff-60d8-4ebd-8f68-6d260b4a2b27 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:46.655487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f3747-ece9-4f09-83bb-6fce5753b305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:46.656148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f3747-ece9-4f09-83bb-6fce5753b305 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:46.656432 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f3747-ece9-4f09-83bb-6fce5753b305 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:46.656705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f3747-ece9-4f09-83bb-6fce5753b305 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:46.657462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f3747-ece9-4f09-83bb-6fce5753b305 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:46.658863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f3747-ece9-4f09-83bb-6fce5753b305 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:46.674157 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 474/923] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jul 12 13:29:46 2022] POST /identity/v3/services => generated 241 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:46.674789 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f3747-ece9-4f09-83bb-6fce5753b305 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:46.777329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947beff-60d8-4ebd-8f68-6d260b4a2b27 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:46.806431 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f3747-ece9-4f09-83bb-6fce5753b305 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:46.820389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2683d2-4163-4da0-a182-89d968215107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:46.821087 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2683d2-4163-4da0-a182-89d968215107 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:46.821442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2683d2-4163-4da0-a182-89d968215107 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:46.821907 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2683d2-4163-4da0-a182-89d968215107 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:46.822779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2683d2-4163-4da0-a182-89d968215107 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:46.823212 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2683d2-4163-4da0-a182-89d968215107 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:46.824905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2683d2-4163-4da0-a182-89d968215107 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:46.850492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 450/924] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Jul 12 13:29:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:46.852363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2683d2-4163-4da0-a182-89d968215107 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:46.852928 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58257a5c-9c03-483b-a356-1c3ba99c23ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:46.856654 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58257a5c-9c03-483b-a356-1c3ba99c23ea None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:46.857209 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58257a5c-9c03-483b-a356-1c3ba99c23ea None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:46.857652 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58257a5c-9c03-483b-a356-1c3ba99c23ea None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:46.858624 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58257a5c-9c03-483b-a356-1c3ba99c23ea None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:46.859113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58257a5c-9c03-483b-a356-1c3ba99c23ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:46.860762 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58257a5c-9c03-483b-a356-1c3ba99c23ea None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:46.868189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 475/925] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jul 12 13:29:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:46.874836 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58257a5c-9c03-483b-a356-1c3ba99c23ea None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:46.878579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:46.924324 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:46.977414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2683d2-4163-4da0-a182-89d968215107 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:47.008201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58257a5c-9c03-483b-a356-1c3ba99c23ea None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:47.015546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19b8ea-ca7e-47fd-b1a3-d9d6663f8cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:47.016355 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19b8ea-ca7e-47fd-b1a3-d9d6663f8cea None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:47.016641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19b8ea-ca7e-47fd-b1a3-d9d6663f8cea None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:47.016915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19b8ea-ca7e-47fd-b1a3-d9d6663f8cea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:47.028923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19b8ea-ca7e-47fd-b1a3-d9d6663f8cea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:47.030447 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19b8ea-ca7e-47fd-b1a3-d9d6663f8cea None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:47.049654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-9a19b8ea-ca7e-47fd-b1a3-d9d6663f8cea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:47.049654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:47.049654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:47.049654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:47.049654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:47.049654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:47.049654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:47.049654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:47.049654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:47.049654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:47.049654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:47.049654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:47.049654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:47.049654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:47.049654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:47.050610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:47.050610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:47.050610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:47.050610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:47.050610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:47.050610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:47.050610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:47.050610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:47.050610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:47.050610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:47.050610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:47.050610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:47.050610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:47.050610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:47.050610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:47.050610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:47.050610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:47.051499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:47.051499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:47.051499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:47.051499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:47.051717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6788ed3-e8b4-4439-9fc3-adc597bceddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:47.051815 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 451/926] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:47.051893 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6788ed3-e8b4-4439-9fc3-adc597bceddb None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:47.051893 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6788ed3-e8b4-4439-9fc3-adc597bceddb None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:47.052488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6788ed3-e8b4-4439-9fc3-adc597bceddb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:47.053986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6788ed3-e8b4-4439-9fc3-adc597bceddb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:47.058467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6788ed3-e8b4-4439-9fc3-adc597bceddb None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:47.060497 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19b8ea-ca7e-47fd-b1a3-d9d6663f8cea None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:47.080201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 476/927] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jul 12 13:29:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:29:47.081735 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4f0ce-1576-4f1a-9cae-54b640a63c12 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:47.082058 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4f0ce-1576-4f1a-9cae-54b640a63c12 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:47.082403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4f0ce-1576-4f1a-9cae-54b640a63c12 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:47.084297 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 477/928] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:47.092671 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a0a59-4e4f-4611-8992-c5ae13b59160 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:47.093084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a0a59-4e4f-4611-8992-c5ae13b59160 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:47.093470 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a0a59-4e4f-4611-8992-c5ae13b59160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:47.116304 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d15a0a59-4e4f-4611-8992-c5ae13b59160 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:47.117887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d15a0a59-4e4f-4611-8992-c5ae13b59160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:47.182070 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19b8ea-ca7e-47fd-b1a3-d9d6663f8cea None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:47.193587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:47.201726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:47.223843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 478/929] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:29:47.236154 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf5fb8-de8f-4cff-8a97-6e13f9a2f5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:47.237140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5fb8-de8f-4cff-8a97-6e13f9a2f5de None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:47.237588 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5fb8-de8f-4cff-8a97-6e13f9a2f5de None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:47.238128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5fb8-de8f-4cff-8a97-6e13f9a2f5de None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:47.239024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf5fb8-de8f-4cff-8a97-6e13f9a2f5de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:47.239520 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf5fb8-de8f-4cff-8a97-6e13f9a2f5de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:47.241118 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf5fb8-de8f-4cff-8a97-6e13f9a2f5de None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:47.252366 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a0a59-4e4f-4611-8992-c5ae13b59160 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:47.259620 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 452/930] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:47.261163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf5fb8-de8f-4cff-8a97-6e13f9a2f5de None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:47.374054 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a0a59-4e4f-4611-8992-c5ae13b59160 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:47.389493 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf5fb8-de8f-4cff-8a97-6e13f9a2f5de None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:47.420408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e0759-f452-4600-82f4-925be8bdb2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:47.421639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e0759-f452-4600-82f4-925be8bdb2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:47.422067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e0759-f452-4600-82f4-925be8bdb2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:47.422487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e0759-f452-4600-82f4-925be8bdb2c3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:47.423871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e0759-f452-4600-82f4-925be8bdb2c3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:47.425732 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e0759-f452-4600-82f4-925be8bdb2c3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:47.434266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35c0b4-c244-41bb-936f-626fcccb0dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:47.435321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35c0b4-c244-41bb-936f-626fcccb0dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:47.435808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35c0b4-c244-41bb-936f-626fcccb0dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:47.436252 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35c0b4-c244-41bb-936f-626fcccb0dd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:47.441378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-d15e0759-f452-4600-82f4-925be8bdb2c3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:29:47.441378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:47.441378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:47.441378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:47.441378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:47.441378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:47.441378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:47.441378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:47.441378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:29:47.441378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:47.441378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:47.441378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:47.441378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:47.441378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:47.441378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:47.441378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:47.442404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:47.442404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:47.442404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:47.442404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:47.442404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:47.442404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:47.442404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:47.442404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:47.442404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:47.442404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:47.442404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:47.442404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:47.442404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:47.442404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:47.442404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:47.442404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:47.443235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:47.443235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:47.443235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:47.443235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:47.443235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:47.443235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:47.443235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:47.443235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:47.443235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:47.443235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:47.443235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:47.443235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:47.443235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:47.443235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:29:47.443235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:47.443235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 479/931] 10.222.0.52 () {56 vars in 1132 bytes} [Tue Jul 12 13:29:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:47.450920 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35c0b4-c244-41bb-936f-626fcccb0dd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:47.452375 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35c0b4-c244-41bb-936f-626fcccb0dd6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:47.453633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e0759-f452-4600-82f4-925be8bdb2c3 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:47.470146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-2c35c0b4-c244-41bb-936f-626fcccb0dd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:47.470146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:47.470146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:47.470146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:47.470146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:47.470146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:47.470146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:47.470146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:47.470146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:47.470146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:47.470146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:47.470146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:47.470146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:47.470146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:47.470146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:47.471101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:47.471101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:47.471101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:47.471101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:47.471101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:47.471101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:47.471101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:47.471101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:47.471101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:47.471101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:47.471101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:47.471101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:47.471101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:47.471101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:47.471101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:47.471101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:47.471101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:47.472182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:47.472182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:47.472182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:47.472182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:47.472182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:47.472182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:47.472182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:47.472182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:47.472182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 453/932] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:47.478173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35c0b4-c244-41bb-936f-626fcccb0dd6 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:47.513826 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:47.520468 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:47.581658 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e0759-f452-4600-82f4-925be8bdb2c3 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:47.597680 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35c0b4-c244-41bb-936f-626fcccb0dd6 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:47.619927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9224769-5f73-410c-81f5-6b045c948a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:47.620874 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9224769-5f73-410c-81f5-6b045c948a65 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:47.621312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9224769-5f73-410c-81f5-6b045c948a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:47.621746 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9224769-5f73-410c-81f5-6b045c948a65 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:47.622739 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9224769-5f73-410c-81f5-6b045c948a65 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:47.623215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9224769-5f73-410c-81f5-6b045c948a65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:47.624877 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9224769-5f73-410c-81f5-6b045c948a65 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:47.631026 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 480/933] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jul 12 13:29:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:47.636856 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9224769-5f73-410c-81f5-6b045c948a65 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:47.638702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21877e4a-debb-458a-aa16-29def509c344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:47.639361 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21877e4a-debb-458a-aa16-29def509c344 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:47.639638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21877e4a-debb-458a-aa16-29def509c344 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:47.639907 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21877e4a-debb-458a-aa16-29def509c344 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:47.640689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21877e4a-debb-458a-aa16-29def509c344 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:47.640992 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21877e4a-debb-458a-aa16-29def509c344 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:47.642482 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21877e4a-debb-458a-aa16-29def509c344 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:47.655841 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 454/934] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:47.662795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21877e4a-debb-458a-aa16-29def509c344 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:47.762341 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:47.763560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9224769-5f73-410c-81f5-6b045c948a65 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:47.775838 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21877e4a-debb-458a-aa16-29def509c344 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:47.780491 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:47.802949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7c7ec-299f-4fca-8b90-9dafc51b0d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:47.803624 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7c7ec-299f-4fca-8b90-9dafc51b0d6b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:47.803911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7c7ec-299f-4fca-8b90-9dafc51b0d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:47.804189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7c7ec-299f-4fca-8b90-9dafc51b0d6b None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:47.805037 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7c7ec-299f-4fca-8b90-9dafc51b0d6b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:47.805293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7c7ec-299f-4fca-8b90-9dafc51b0d6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:47.806913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7c7ec-299f-4fca-8b90-9dafc51b0d6b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:47.813255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 481/935] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jul 12 13:29:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:47.819380 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724a92e-3a16-4f11-8e15-5fb2b5c6cfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:47.820285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724a92e-3a16-4f11-8e15-5fb2b5c6cfd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:47.820569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724a92e-3a16-4f11-8e15-5fb2b5c6cfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:47.820847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724a92e-3a16-4f11-8e15-5fb2b5c6cfd0 None admin] PATH_INFO: `/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/dedf564ba5744a1fa70bb78222d26d5b/roles/19deffe64f224b68880ce003b4af8b2b` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:47.823511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7c7ec-299f-4fca-8b90-9dafc51b0d6b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:47.823786 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724a92e-3a16-4f11-8e15-5fb2b5c6cfd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=36aa0326fd0545dface07a9ddc308ccd, user_id=dedf564ba5744a1fa70bb78222d26d5b, role_id=19deffe64f224b68880ce003b4af8b2b)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:47.825371 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724a92e-3a16-4f11-8e15-5fb2b5c6cfd0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:47.843513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 455/936] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:29:47 2022] PUT /identity/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/dedf564ba5744a1fa70bb78222d26d5b/roles/19deffe64f224b68880ce003b4af8b2b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:29:47.952231 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7c7ec-299f-4fca-8b90-9dafc51b0d6b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:47.992400 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2ceaf-c24a-4b3a-a932-14c5d1fbffad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:47.993371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2ceaf-c24a-4b3a-a932-14c5d1fbffad None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:47.993809 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2ceaf-c24a-4b3a-a932-14c5d1fbffad None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:47.994262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2ceaf-c24a-4b3a-a932-14c5d1fbffad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:47.995731 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2ceaf-c24a-4b3a-a932-14c5d1fbffad None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:47.997281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2ceaf-c24a-4b3a-a932-14c5d1fbffad None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:48.021378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 482/937] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jul 12 13:29:47 2022] POST /identity/v3/endpoints => generated 347 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:48.908973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f9097-8d43-43dc-98bc-4540656ef619 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:48.909527 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f9097-8d43-43dc-98bc-4540656ef619 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:48.909527 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f9097-8d43-43dc-98bc-4540656ef619 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:48.910936 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 456/938] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:29:48.916391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09ba65-156c-439a-a2d3-ec7530c7f6d6 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:48.917009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09ba65-156c-439a-a2d3-ec7530c7f6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:48.917009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09ba65-156c-439a-a2d3-ec7530c7f6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:48.934751 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1a09ba65-156c-439a-a2d3-ec7530c7f6d6 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:48.935956 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1a09ba65-156c-439a-a2d3-ec7530c7f6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:48.968284 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:49.026571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 483/939] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:49.046380 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f9097-8d43-43dc-98bc-4540656ef619 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:49.153542 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f9097-8d43-43dc-98bc-4540656ef619 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:49.188839 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e593b-d59f-488e-b956-814f3dbebfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:49.189660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e593b-d59f-488e-b956-814f3dbebfd0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.189946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e593b-d59f-488e-b956-814f3dbebfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.190253 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e593b-d59f-488e-b956-814f3dbebfd0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.190917 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e593b-d59f-488e-b956-814f3dbebfd0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:49.192328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e593b-d59f-488e-b956-814f3dbebfd0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:49.205848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-3a0e593b-d59f-488e-b956-814f3dbebfd0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:29:49.205848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:49.205848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:49.205848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:49.205848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:49.205848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:49.205848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:49.205848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:49.205848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:29:49.205848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:49.205848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:49.205848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:49.205848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:49.205848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:49.205848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:49.205848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:49.205848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:49.206947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:49.206947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:49.206947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:49.206947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:49.206947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:49.206947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:49.206947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:49.206947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:49.206947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:49.206947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:49.206947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:49.206947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:49.206947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:49.206947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:49.206947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:49.206947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:49.207788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:49.207788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:49.207788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:49.207788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:49.207788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:49.207788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:49.207788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:49.207788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:49.207788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:49.207788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:49.207788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:49.207788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:49.207788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:29:49.207788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:49.212204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 457/940] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jul 12 13:29:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:49.214113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09ba65-156c-439a-a2d3-ec7530c7f6d6 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:49.301514 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e795b-0642-4bdf-be24-93a1bdc86d31 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.301841 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e795b-0642-4bdf-be24-93a1bdc86d31 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.302106 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e795b-0642-4bdf-be24-93a1bdc86d31 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.303595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 458/941] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:29:49.308828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5073b3-1f8c-43dd-bfce-ffabea02980f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.309104 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5073b3-1f8c-43dd-bfce-ffabea02980f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.309383 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5073b3-1f8c-43dd-bfce-ffabea02980f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.325353 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09ba65-156c-439a-a2d3-ec7530c7f6d6 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:49.332850 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9b5073b3-1f8c-43dd-bfce-ffabea02980f None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:49.334744 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9b5073b3-1f8c-43dd-bfce-ffabea02980f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:49.350605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:49.374742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160932f5-4a55-4ea6-8550-5e09305d10b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:49.375921 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160932f5-4a55-4ea6-8550-5e09305d10b3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.376714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160932f5-4a55-4ea6-8550-5e09305d10b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.377274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160932f5-4a55-4ea6-8550-5e09305d10b3 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.378198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160932f5-4a55-4ea6-8550-5e09305d10b3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:49.378683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160932f5-4a55-4ea6-8550-5e09305d10b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:49.381251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160932f5-4a55-4ea6-8550-5e09305d10b3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:49.387737 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 484/942] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jul 12 13:29:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:49.394964 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160932f5-4a55-4ea6-8550-5e09305d10b3 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:49.422074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 459/943] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:49.433234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:49.442568 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5073b3-1f8c-43dd-bfce-ffabea02980f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:49.535175 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160932f5-4a55-4ea6-8550-5e09305d10b3 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:49.569947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5073b3-1f8c-43dd-bfce-ffabea02980f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:49.579235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a2693-a971-4ba7-b8f0-a2b632d92c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:49.580711 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a2693-a971-4ba7-b8f0-a2b632d92c65 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.581195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a2693-a971-4ba7-b8f0-a2b632d92c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.581657 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a2693-a971-4ba7-b8f0-a2b632d92c65 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.582960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a2693-a971-4ba7-b8f0-a2b632d92c65 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:49.583852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a2693-a971-4ba7-b8f0-a2b632d92c65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:49.585679 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a2693-a971-4ba7-b8f0-a2b632d92c65 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:49.598565 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 485/944] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jul 12 13:29:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:49.600105 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a2693-a971-4ba7-b8f0-a2b632d92c65 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:49.615390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc74774-2047-4f09-a22b-26f536b87b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:49.616166 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc74774-2047-4f09-a22b-26f536b87b2a None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.616836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc74774-2047-4f09-a22b-26f536b87b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.617197 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc74774-2047-4f09-a22b-26f536b87b2a None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.618446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc74774-2047-4f09-a22b-26f536b87b2a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:49.620196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc74774-2047-4f09-a22b-26f536b87b2a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:49.622334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: WARNING keystone.server.flask.common [None req-0fc74774-2047-4f09-a22b-26f536b87b2a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 12 13:29:49.640824 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:49.647728 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.sql.core [None req-0fc74774-2047-4f09-a22b-26f536b87b2a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 13:29:49.647728 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 12 13:29:49.647728 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [parameters: {'id': '610d504ae59e422dbc12681d2088184c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 12 13:29:49.647728 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80500) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 13:29:49.657149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-0fc74774-2047-4f09-a22b-26f536b87b2a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 610d504ae59e422dbc12681d2088184c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 610d504ae59e422dbc12681d2088184c. Jul 12 13:29:49.657149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:49.657149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 13:29:49.657149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 13:29:49.657149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 13:29:49.657149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 13:29:49.657149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 13:29:49.657149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 13:29:49.657149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 13:29:49.657149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application conn.query(q) Jul 12 13:29:49.657149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 13:29:49.657149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 13:29:49.657149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application result.read() Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 12 13:29:49.657952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 12 13:29:49.659054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 13:29:49.659054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application next(self.gen) Jul 12 13:29:49.659054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 13:29:49.659054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application yield resource Jul 12 13:29:49.659054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 13:29:49.659054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application next(self.gen) Jul 12 13:29:49.659054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 13:29:49.659054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 13:29:49.659054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 13:29:49.659054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 13:29:49.659054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 13:29:49.659054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise self.value Jul 12 13:29:49.659054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 13:29:49.659054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 13:29:49.659054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 13:29:49.659054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application session.commit() Jul 12 13:29:49.659054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 13:29:49.659927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 13:29:49.659927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 13:29:49.659927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 13:29:49.659927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 13:29:49.659927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self.session.flush() Jul 12 13:29:49.659927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 13:29:49.659927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 13:29:49.659927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 13:29:49.659927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 13:29:49.659927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 13:29:49.659927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application compat.raise_( Jul 12 13:29:49.659927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 13:29:49.659927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception Jul 12 13:29:49.659927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 13:29:49.659927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 13:29:49.659927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 13:29:49.659927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 13:29:49.660832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 13:29:49.660832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 13:29:49.660832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 13:29:49.660832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 13:29:49.660832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 12 13:29:49.660832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 12 13:29:49.660832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 13:29:49.660832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 13:29:49.660832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 13:29:49.660832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 13:29:49.660832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 13:29:49.660832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 13:29:49.660832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 13:29:49.660832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 13:29:49.660832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 13:29:49.661599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 13:29:49.661599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 13:29:49.661599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception Jul 12 13:29:49.661599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 13:29:49.661599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 13:29:49.661599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 13:29:49.661599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 13:29:49.661599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 13:29:49.661599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 13:29:49.661599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 13:29:49.661599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application conn.query(q) Jul 12 13:29:49.661599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 13:29:49.661599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 13:29:49.661599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 13:29:49.661599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application result.read() Jul 12 13:29:49.661599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 13:29:49.661599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 13:29:49.662679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 13:29:49.662679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 13:29:49.662679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 13:29:49.662679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 13:29:49.662679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 13:29:49.662679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 13:29:49.662679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 13:29:49.662679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 12 13:29:49.662679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [parameters: {'id': '610d504ae59e422dbc12681d2088184c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 12 13:29:49.662679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 13:29:49.662679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:49.662679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:49.662679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:49.662679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:49.662679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 12 13:29:49.663621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 12 13:29:49.663621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 13:29:49.663621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 13:29:49.663621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 12 13:29:49.663621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:49.663621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:49.663621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:49.663621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:49.663621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:49.663621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:49.663621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:49.663621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:49.663621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:49.663621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:49.663621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:49.663621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:49.663621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:49.665806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:49.665806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 12 13:29:49.665806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 12 13:29:49.665806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:49.665806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:49.665806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 12 13:29:49.665806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 12 13:29:49.665806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 610d504ae59e422dbc12681d2088184c. Jul 12 13:29:49.665806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:49.673817 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 460/945] 10.222.0.52 () {60 vars in 1140 bytes} [Tue Jul 12 13:29:49 2022] POST /identity/v3/projects => generated 262 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:49.675363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc74774-2047-4f09-a22b-26f536b87b2a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:49.705660 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:49.720396 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a2693-a971-4ba7-b8f0-a2b632d92c65 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:49.755601 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd79633-bd0f-4ae5-a746-5f5363fd028a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:49.756397 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd79633-bd0f-4ae5-a746-5f5363fd028a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.756753 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd79633-bd0f-4ae5-a746-5f5363fd028a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.757118 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd79633-bd0f-4ae5-a746-5f5363fd028a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.758006 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd79633-bd0f-4ae5-a746-5f5363fd028a None admin] RBAC: Adding query filter params (interface=public, service_id=ac4a0f831dfc4cf7a4d650d7dfffb590, region_id=RegionOne) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:49.758391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd79633-bd0f-4ae5-a746-5f5363fd028a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:49.759859 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd79633-bd0f-4ae5-a746-5f5363fd028a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:49.770010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 486/946] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Jul 12 13:29:49 2022] GET /identity/v3/endpoints?service_id=ac4a0f831dfc4cf7a4d650d7dfffb590&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:49.772539 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd79633-bd0f-4ae5-a746-5f5363fd028a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:49.803239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc74774-2047-4f09-a22b-26f536b87b2a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:49.856048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b26d7-8db2-4b6c-ad41-300900780198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:49.856865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b26d7-8db2-4b6c-ad41-300900780198 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.857208 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b26d7-8db2-4b6c-ad41-300900780198 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.857490 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b26d7-8db2-4b6c-ad41-300900780198 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.866453 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b26d7-8db2-4b6c-ad41-300900780198 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:49.868998 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b26d7-8db2-4b6c-ad41-300900780198 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:49.888913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-e99b26d7-8db2-4b6c-ad41-300900780198 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:49.888913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:49.888913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:49.888913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:49.888913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:49.888913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:49.888913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:49.888913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:49.888913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:49.888913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:49.888913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:49.888913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:49.888913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:49.888913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:49.888913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:49.889939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:49.889939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:49.889939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:49.889939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:49.889939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:49.889939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:49.889939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:49.889939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:49.889939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:49.889939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:49.889939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:49.889939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:49.889939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:49.889939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:49.889939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:49.889939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:49.889939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:49.890967 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:49.890967 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:49.890967 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:49.890967 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:49.890967 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:49.890967 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:49.890967 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:49.890967 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:49.894042 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 461/947] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:49.897164 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b26d7-8db2-4b6c-ad41-300900780198 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:49.911854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd79633-bd0f-4ae5-a746-5f5363fd028a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:49.962416 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d4545-d87e-40c7-90dc-b8df84572686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:49.963018 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d4545-d87e-40c7-90dc-b8df84572686 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.963264 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d4545-d87e-40c7-90dc-b8df84572686 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.963579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d4545-d87e-40c7-90dc-b8df84572686 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.964257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d4545-d87e-40c7-90dc-b8df84572686 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:49.965657 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d4545-d87e-40c7-90dc-b8df84572686 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:49.972012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 487/948] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jul 12 13:29:49 2022] GET /identity/v3/services => generated 1497 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:29:49.973260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efab7c-b6a1-449b-8b1a-166ad59b0a65 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.973547 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efab7c-b6a1-449b-8b1a-166ad59b0a65 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.973822 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efab7c-b6a1-449b-8b1a-166ad59b0a65 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.975294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 488/949] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:49.983757 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d7089-3465-4c01-8d92-53bd3f9db4aa None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.984303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d7089-3465-4c01-8d92-53bd3f9db4aa None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.984790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d7089-3465-4c01-8d92-53bd3f9db4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:50.001865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:50.003824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8b3d7089-3465-4c01-8d92-53bd3f9db4aa None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:50.005052 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8b3d7089-3465-4c01-8d92-53bd3f9db4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:50.018817 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:50.039766 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b26d7-8db2-4b6c-ad41-300900780198 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:50.087727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c8143-898d-4a98-ae55-c8c9652e87d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:50.089194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c8143-898d-4a98-ae55-c8c9652e87d2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:50.089473 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 489/950] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:29:50.090354 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c8143-898d-4a98-ae55-c8c9652e87d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:50.090641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c8143-898d-4a98-ae55-c8c9652e87d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:50.091430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c8143-898d-4a98-ae55-c8c9652e87d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:50.091742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c8143-898d-4a98-ae55-c8c9652e87d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:50.093217 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c8143-898d-4a98-ae55-c8c9652e87d2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:50.116026 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d7089-3465-4c01-8d92-53bd3f9db4aa None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:50.117301 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 462/951] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jul 12 13:29:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:50.175600 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533abed-bdb7-4584-bd6f-b2ffbd0a2b4d None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:50.175975 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533abed-bdb7-4584-bd6f-b2ffbd0a2b4d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:50.176161 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533abed-bdb7-4584-bd6f-b2ffbd0a2b4d None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:50.177940 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 463/952] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:29:50.183700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83144c-1a1d-490a-bf0b-ac6027d3d531 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:50.184004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83144c-1a1d-490a-bf0b-ac6027d3d531 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:50.184261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83144c-1a1d-490a-bf0b-ac6027d3d531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:50.203292 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-5a83144c-1a1d-490a-bf0b-ac6027d3d531 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:50.205297 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5a83144c-1a1d-490a-bf0b-ac6027d3d531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:50.238662 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d7089-3465-4c01-8d92-53bd3f9db4aa None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:50.279606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752c9ca-43d0-431f-9fdc-a4b858e222b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:50.279606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752c9ca-43d0-431f-9fdc-a4b858e222b6 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:50.279606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752c9ca-43d0-431f-9fdc-a4b858e222b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:50.279606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752c9ca-43d0-431f-9fdc-a4b858e222b6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:50.288695 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 464/953] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:50.298771 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752c9ca-43d0-431f-9fdc-a4b858e222b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:29:50.307482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-2752c9ca-43d0-431f-9fdc-a4b858e222b6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:29:50.307482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:50.307482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:50.307482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:50.307482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:50.307482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:50.307482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:50.307482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:50.307482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:50.307482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:50.307482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:50.307482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:50.307482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:29:50.307482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:29:50.307482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:29:50.307482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:29:50.308656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:29:50.308656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:29:50.308656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:29:50.308656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:29:50.308656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:50.308656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:50.308656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:50.308656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:50.308656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:50.308656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:50.308656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:50.308656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:50.308656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:50.308656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:50.308656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:50.308656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:50.309736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:50.309736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:50.309736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:50.309736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:50.309736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:29:50.309736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:29:50.309736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:29:50.309736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:29:50.309736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:29:50.309736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:29:50.309736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:29:50.309736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:50.311706 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 490/954] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Jul 12 13:29:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:50.314174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83144c-1a1d-490a-bf0b-ac6027d3d531 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:50.316810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752c9ca-43d0-431f-9fdc-a4b858e222b6 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:50.330815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:50.373240 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:50.434104 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83144c-1a1d-490a-bf0b-ac6027d3d531 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:50.454233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752c9ca-43d0-431f-9fdc-a4b858e222b6 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:50.471747 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a20b2c-1a21-4185-a828-5fc107569ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:50.472563 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a20b2c-1a21-4185-a828-5fc107569ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:50.472848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a20b2c-1a21-4185-a828-5fc107569ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:50.473127 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a20b2c-1a21-4185-a828-5fc107569ca3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:50.473786 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a20b2c-1a21-4185-a828-5fc107569ca3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:50.475206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a20b2c-1a21-4185-a828-5fc107569ca3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:50.488063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-b4a20b2c-1a21-4185-a828-5fc107569ca3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:29:50.488063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:50.488063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:50.488063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:50.488063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:50.488063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:50.488063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:50.488063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:50.488063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:29:50.488063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:50.488063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:50.488063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:50.488063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:50.488063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:50.488063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:50.488063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:50.488063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:50.489196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:50.489196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:50.489196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:50.489196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:50.489196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:50.489196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:50.489196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:50.489196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:50.489196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:50.489196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:50.489196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:50.489196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:50.489196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:50.489196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:50.489196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:50.489196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:50.490261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:50.490261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:50.490261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:50.490261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:50.490261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:50.490261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:50.490261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:50.490261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:50.490261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:50.490261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:50.490261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:50.490261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:50.490261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:29:50.490261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:50.490261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 465/955] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:50.495484 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a20b2c-1a21-4185-a828-5fc107569ca3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:50.504092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65cfea-e978-40fc-84b8-6425d87bfe29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:50.505044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65cfea-e978-40fc-84b8-6425d87bfe29 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:50.505517 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65cfea-e978-40fc-84b8-6425d87bfe29 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:50.505996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65cfea-e978-40fc-84b8-6425d87bfe29 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:50.507401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65cfea-e978-40fc-84b8-6425d87bfe29 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:50.507910 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65cfea-e978-40fc-84b8-6425d87bfe29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:50.510704 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65cfea-e978-40fc-84b8-6425d87bfe29 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:50.522009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 491/956] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Jul 12 13:29:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:50.527270 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65cfea-e978-40fc-84b8-6425d87bfe29 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:50.603534 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:50.615758 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a20b2c-1a21-4185-a828-5fc107569ca3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:50.639680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65cfea-e978-40fc-84b8-6425d87bfe29 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:50.646326 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:50.657177 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089d853-7fdc-4950-99b4-b30db8451f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:50.658812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089d853-7fdc-4950-99b4-b30db8451f4c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:50.659550 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089d853-7fdc-4950-99b4-b30db8451f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:50.661609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089d853-7fdc-4950-99b4-b30db8451f4c None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:50.663058 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089d853-7fdc-4950-99b4-b30db8451f4c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:50.663922 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089d853-7fdc-4950-99b4-b30db8451f4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:50.666586 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089d853-7fdc-4950-99b4-b30db8451f4c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:50.682621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6033f-b612-4153-814e-68cd67f1e8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:50.682621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6033f-b612-4153-814e-68cd67f1e8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:50.682621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6033f-b612-4153-814e-68cd67f1e8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:50.682621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6033f-b612-4153-814e-68cd67f1e8a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:50.683321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 466/957] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jul 12 13:29:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:50.686477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089d853-7fdc-4950-99b4-b30db8451f4c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:50.695579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6033f-b612-4153-814e-68cd67f1e8a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:50.697330 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6033f-b612-4153-814e-68cd67f1e8a2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:50.716876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-2ea6033f-b612-4153-814e-68cd67f1e8a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:50.716876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:50.716876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:50.716876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:50.716876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:50.716876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:50.716876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:50.716876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:50.716876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:50.716876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:50.716876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:50.716876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:50.716876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:50.716876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:50.716876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:50.717835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:50.717835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:50.717835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:50.717835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:50.717835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:50.717835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:50.717835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:50.717835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:50.717835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:50.717835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:50.717835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:50.717835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:50.717835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:50.717835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:50.717835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:50.717835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:50.717835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:50.718772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:50.718772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:50.718772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:50.718772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:50.725231 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 492/958] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:50.726591 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6033f-b612-4153-814e-68cd67f1e8a2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:50.805204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089d853-7fdc-4950-99b4-b30db8451f4c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:50.839153 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6033f-b612-4153-814e-68cd67f1e8a2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:50.844915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754aa1d-a85b-43d6-a412-d8d4fbe84417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:50.845807 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754aa1d-a85b-43d6-a412-d8d4fbe84417 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:50.846245 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754aa1d-a85b-43d6-a412-d8d4fbe84417 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:50.846619 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754aa1d-a85b-43d6-a412-d8d4fbe84417 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:50.847547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754aa1d-a85b-43d6-a412-d8d4fbe84417 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:50.847956 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754aa1d-a85b-43d6-a412-d8d4fbe84417 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:50.849541 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754aa1d-a85b-43d6-a412-d8d4fbe84417 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:50.862375 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 467/959] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jul 12 13:29:50 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:50.875552 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:50.892926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf1c6b-7c7a-4304-b1b7-6e71c9f41884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:50.893654 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf1c6b-7c7a-4304-b1b7-6e71c9f41884 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:50.893950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf1c6b-7c7a-4304-b1b7-6e71c9f41884 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:50.894256 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf1c6b-7c7a-4304-b1b7-6e71c9f41884 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:50.894990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf1c6b-7c7a-4304-b1b7-6e71c9f41884 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:50.895298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf1c6b-7c7a-4304-b1b7-6e71c9f41884 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:50.896709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf1c6b-7c7a-4304-b1b7-6e71c9f41884 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:50.911883 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 493/960] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:50.917647 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754aa1d-a85b-43d6-a412-d8d4fbe84417 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:50.970241 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:51.059221 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754aa1d-a85b-43d6-a412-d8d4fbe84417 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:51.094770 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd7377-91df-4ab9-960c-ea18c3ba09a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:51.095585 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd7377-91df-4ab9-960c-ea18c3ba09a1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:51.095868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd7377-91df-4ab9-960c-ea18c3ba09a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:51.096145 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd7377-91df-4ab9-960c-ea18c3ba09a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:51.104735 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd7377-91df-4ab9-960c-ea18c3ba09a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:51.106332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd7377-91df-4ab9-960c-ea18c3ba09a1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:51.122971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-bdbd7377-91df-4ab9-960c-ea18c3ba09a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:29:51.122971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:51.122971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:51.122971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:51.122971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:51.122971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:51.122971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:51.122971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:51.122971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:51.122971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:51.122971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:51.122971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:51.122971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:51.122971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:51.122971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:51.122971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:51.126577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:51.127599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:51.127599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:51.127599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:51.127599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:51.127599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:51.127599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:51.127599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:51.127599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:51.127599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:29:51.127599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:51.127599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 468/961] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jul 12 13:29:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:51.131956 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf1c6b-7c7a-4304-b1b7-6e71c9f41884 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:51.239151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf1c6b-7c7a-4304-b1b7-6e71c9f41884 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:51.275493 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4695a-c8d2-41ce-afd7-22bf7ac6e367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:51.276205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4695a-c8d2-41ce-afd7-22bf7ac6e367 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:51.276490 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4695a-c8d2-41ce-afd7-22bf7ac6e367 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:51.276769 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4695a-c8d2-41ce-afd7-22bf7ac6e367 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:51.277613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4695a-c8d2-41ce-afd7-22bf7ac6e367 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:51.277922 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4695a-c8d2-41ce-afd7-22bf7ac6e367 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:51.279450 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4695a-c8d2-41ce-afd7-22bf7ac6e367 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:51.299173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 494/962] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Jul 12 13:29:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:51.304672 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd7377-91df-4ab9-960c-ea18c3ba09a1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:51.413606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:51.417141 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd7377-91df-4ab9-960c-ea18c3ba09a1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:51.453895 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bd479-dc77-4646-93ba-c6ae37ff71f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:51.454710 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bd479-dc77-4646-93ba-c6ae37ff71f5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:51.454997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bd479-dc77-4646-93ba-c6ae37ff71f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:51.455270 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bd479-dc77-4646-93ba-c6ae37ff71f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:51.527891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7bd479-dc77-4646-93ba-c6ae37ff71f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:51.529496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7bd479-dc77-4646-93ba-c6ae37ff71f5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:51.628255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:51.632096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 469/963] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:29:51 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:29:51.638331 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4695a-c8d2-41ce-afd7-22bf7ac6e367 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:51.683683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:51.744334 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4695a-c8d2-41ce-afd7-22bf7ac6e367 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:51.784963 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be79e0-8f2d-4a5a-b16c-08ad07a87bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:51.786159 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be79e0-8f2d-4a5a-b16c-08ad07a87bfe None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:51.786649 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be79e0-8f2d-4a5a-b16c-08ad07a87bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:51.787195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be79e0-8f2d-4a5a-b16c-08ad07a87bfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:51.800157 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be79e0-8f2d-4a5a-b16c-08ad07a87bfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:51.802853 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be79e0-8f2d-4a5a-b16c-08ad07a87bfe None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:51.829815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-e9be79e0-8f2d-4a5a-b16c-08ad07a87bfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:51.829815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:51.829815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:51.829815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:51.829815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:51.829815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:51.829815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:51.829815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:51.829815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:51.829815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:51.829815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:51.829815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:51.829815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:29:51.829815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:29:51.829815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:29:51.830888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:29:51.830888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:51.830888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:51.830888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:51.830888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:51.830888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:51.830888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:51.830888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:51.830888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:51.830888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:51.830888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:51.830888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:51.830888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:51.830888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:51.830888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:51.830888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:51.830888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:51.831871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:29:51.831871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:29:51.831871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:29:51.831871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:51.834140 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 495/964] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:51.844653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bd479-dc77-4646-93ba-c6ae37ff71f5 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:51.956973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bd479-dc77-4646-93ba-c6ae37ff71f5 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:51.992683 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051b407-24ff-49e4-ae2f-7abbc380d1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:51.994059 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051b407-24ff-49e4-ae2f-7abbc380d1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:51.995099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051b407-24ff-49e4-ae2f-7abbc380d1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:51.995579 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051b407-24ff-49e4-ae2f-7abbc380d1d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:51.996508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051b407-24ff-49e4-ae2f-7abbc380d1d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:51.997043 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051b407-24ff-49e4-ae2f-7abbc380d1d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:51.998641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051b407-24ff-49e4-ae2f-7abbc380d1d8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:52.011956 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 470/965] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:52.017786 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be79e0-8f2d-4a5a-b16c-08ad07a87bfe None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:52.089913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324cfca-3b96-45ec-9d07-949adb887d20 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:52.090474 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324cfca-3b96-45ec-9d07-949adb887d20 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:52.090554 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324cfca-3b96-45ec-9d07-949adb887d20 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:52.092141 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 471/966] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:29:52.097156 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d464067-c7f4-46e9-af6a-f4101d6f9f76 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:52.097452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d464067-c7f4-46e9-af6a-f4101d6f9f76 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:52.097736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d464067-c7f4-46e9-af6a-f4101d6f9f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:52.116168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1d464067-c7f4-46e9-af6a-f4101d6f9f76 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:52.117453 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1d464067-c7f4-46e9-af6a-f4101d6f9f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:52.128147 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:52.144310 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be79e0-8f2d-4a5a-b16c-08ad07a87bfe None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:52.189105 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc6d3e-9d83-4224-a4b0-765f06e0967a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:52.190461 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc6d3e-9d83-4224-a4b0-765f06e0967a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:52.191240 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc6d3e-9d83-4224-a4b0-765f06e0967a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:52.191719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc6d3e-9d83-4224-a4b0-765f06e0967a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:52.197697 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:52.201201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc6d3e-9d83-4224-a4b0-765f06e0967a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:52.203424 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc6d3e-9d83-4224-a4b0-765f06e0967a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:52.218310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 472/967] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:52.220488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-d6fc6d3e-9d83-4224-a4b0-765f06e0967a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:52.220488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:52.220488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:52.220488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:52.220488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:52.220488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:52.220488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:52.220488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:52.220488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:52.220488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:52.220488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:52.220488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:52.220488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:29:52.220488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:29:52.220488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:29:52.221343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:29:52.221343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:52.221343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:52.221343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:52.221343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:52.221343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:52.221343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:52.221343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:52.221343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:52.221343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:52.221343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:52.221343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:52.221343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:52.221343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:52.221343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:52.221343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:52.221343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:52.222260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:29:52.222260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:29:52.222260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:29:52.222260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:29:52.222260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:29:52.222260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:29:52.222260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:29:52.222260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:52.224706 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 496/968] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:52.231105 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d464067-c7f4-46e9-af6a-f4101d6f9f76 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:52.241932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc6d3e-9d83-4224-a4b0-765f06e0967a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:52.349580 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d464067-c7f4-46e9-af6a-f4101d6f9f76 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:52.353539 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc6d3e-9d83-4224-a4b0-765f06e0967a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:52.386517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f6f43-fd7c-4b55-b936-6ea5d314e4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:52.387171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f6f43-fd7c-4b55-b936-6ea5d314e4ae None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:52.387448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f6f43-fd7c-4b55-b936-6ea5d314e4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:52.387716 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f6f43-fd7c-4b55-b936-6ea5d314e4ae None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:52.388483 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f6f43-fd7c-4b55-b936-6ea5d314e4ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:52.388837 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f6f43-fd7c-4b55-b936-6ea5d314e4ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:52.389445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db0768-fd55-46ec-b8c8-0113d87d0dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:52.390291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f6f43-fd7c-4b55-b936-6ea5d314e4ae None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:52.390408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db0768-fd55-46ec-b8c8-0113d87d0dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:52.390628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db0768-fd55-46ec-b8c8-0113d87d0dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:52.390905 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db0768-fd55-46ec-b8c8-0113d87d0dd7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:52.391557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db0768-fd55-46ec-b8c8-0113d87d0dd7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:52.393038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db0768-fd55-46ec-b8c8-0113d87d0dd7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:52.402623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 473/969] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:29:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:52.403823 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99d064-b549-42b6-969b-8bfeeb1dd56c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:52.404111 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99d064-b549-42b6-969b-8bfeeb1dd56c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:52.404381 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99d064-b549-42b6-969b-8bfeeb1dd56c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:52.405733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 474/970] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:52.406269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-e4db0768-fd55-46ec-b8c8-0113d87d0dd7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:29:52.406269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:52.406269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:52.406269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:52.406269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:52.406269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:52.406269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:52.406269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:52.406269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:29:52.406269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:52.406269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:52.406269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:52.406269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:52.406269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:52.406269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:52.406269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:52.406269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:52.407274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:52.407274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:52.407274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:52.407274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:52.407274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:52.407274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:52.407274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:52.407274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:52.407274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:52.407274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:52.407274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:52.407274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:52.407274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:52.407274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:52.407274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:52.407274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:52.408126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:52.408126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:52.408126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:52.408126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:52.408126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:52.408126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:52.408126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:52.408126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:52.408126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:52.408126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:52.408126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:52.408126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:52.408126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:29:52.408126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:52.410600 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99d064-b549-42b6-969b-8bfeeb1dd56c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:52.412382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 497/971] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jul 12 13:29:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:52.413670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedbcc5-63a8-4b4b-95f6-a17b033d4a9a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:52.413991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedbcc5-63a8-4b4b-95f6-a17b033d4a9a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:52.415051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedbcc5-63a8-4b4b-95f6-a17b033d4a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:52.432100 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4dedbcc5-63a8-4b4b-95f6-a17b033d4a9a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:52.433300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4dedbcc5-63a8-4b4b-95f6-a17b033d4a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:52.443208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:52.465607 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:52.509886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 498/972] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:52.511111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedbcc5-63a8-4b4b-95f6-a17b033d4a9a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:52.543762 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99d064-b549-42b6-969b-8bfeeb1dd56c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:52.580348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b174ecd-b414-46d1-a974-97bfb95dc82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:52.581289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b174ecd-b414-46d1-a974-97bfb95dc82c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:52.581289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b174ecd-b414-46d1-a974-97bfb95dc82c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:52.581599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b174ecd-b414-46d1-a974-97bfb95dc82c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:52.582401 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b174ecd-b414-46d1-a974-97bfb95dc82c None admin] RBAC: Adding query filter params (role.id=19deffe64f224b68880ce003b4af8b2b, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=dedf564ba5744a1fa70bb78222d26d5b) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:52.582718 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b174ecd-b414-46d1-a974-97bfb95dc82c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:52.584170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b174ecd-b414-46d1-a974-97bfb95dc82c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:52.590974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 475/973] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:29:52 2022] GET /identity/v3/role_assignments?user.id=dedf564ba5744a1fa70bb78222d26d5b&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=19deffe64f224b68880ce003b4af8b2b => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:52.591661 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b174ecd-b414-46d1-a974-97bfb95dc82c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:52.621173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedbcc5-63a8-4b4b-95f6-a17b033d4a9a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:52.664871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19a7cc-5517-46cf-8903-15bd3942b855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:52.665684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19a7cc-5517-46cf-8903-15bd3942b855 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:52.666059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19a7cc-5517-46cf-8903-15bd3942b855 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:52.666464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19a7cc-5517-46cf-8903-15bd3942b855 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:52.667250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19a7cc-5517-46cf-8903-15bd3942b855 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:52.667633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19a7cc-5517-46cf-8903-15bd3942b855 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:52.669135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19a7cc-5517-46cf-8903-15bd3942b855 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:52.676669 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 499/974] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jul 12 13:29:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:52.682966 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19a7cc-5517-46cf-8903-15bd3942b855 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:52.696754 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:52.744065 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b174ecd-b414-46d1-a974-97bfb95dc82c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:52.762997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:52.789195 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6a0c5-eba1-40bf-89c6-ca048282a699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:52.790532 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6a0c5-eba1-40bf-89c6-ca048282a699 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:52.791036 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6a0c5-eba1-40bf-89c6-ca048282a699 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:52.791607 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6a0c5-eba1-40bf-89c6-ca048282a699 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:52.793135 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6a0c5-eba1-40bf-89c6-ca048282a699 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:52.794831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6a0c5-eba1-40bf-89c6-ca048282a699 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:52.813942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.sql.core [None req-f2a6a0c5-eba1-40bf-89c6-ca048282a699 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 13:29:52.813942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 12 13:29:52.813942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [parameters: {'user_id': 'c839bc9c48b3433d8f329d1737f0d4fa', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 12 13:29:52.813942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80500) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 13:29:52.821248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-f2a6a0c5-eba1-40bf-89c6-ca048282a699 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 12 13:29:52.821248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:52.821248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 13:29:52.821248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 13:29:52.821248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 13:29:52.821248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 13:29:52.821248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 13:29:52.821248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 13:29:52.821248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 13:29:52.821248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application conn.query(q) Jul 12 13:29:52.821248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 13:29:52.821248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 13:29:52.821248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 13:29:52.821248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application result.read() Jul 12 13:29:52.821248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 13:29:52.822258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application next(self.gen) Jul 12 13:29:52.823832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 13:29:52.823832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application yield resource Jul 12 13:29:52.823832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 13:29:52.823832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application next(self.gen) Jul 12 13:29:52.823832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 13:29:52.823832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 13:29:52.823832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 13:29:52.823832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 13:29:52.823832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 13:29:52.823832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise self.value Jul 12 13:29:52.823832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 13:29:52.823832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 13:29:52.823832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 13:29:52.823832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application session.commit() Jul 12 13:29:52.823832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 13:29:52.823832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 13:29:52.823832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 13:29:52.824741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 13:29:52.824741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 13:29:52.824741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self.session.flush() Jul 12 13:29:52.824741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 13:29:52.824741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 13:29:52.824741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 13:29:52.824741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 13:29:52.824741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 13:29:52.824741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application compat.raise_( Jul 12 13:29:52.824741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 13:29:52.824741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception Jul 12 13:29:52.824741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 13:29:52.824741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 13:29:52.824741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 13:29:52.824741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 13:29:52.824741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 13:29:52.824741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 13:29:52.825628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 13:29:52.825628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 13:29:52.825628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 12 13:29:52.825628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 12 13:29:52.825628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 13:29:52.825628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 13:29:52.825628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 13:29:52.825628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 13:29:52.825628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 13:29:52.825628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 13:29:52.825628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 13:29:52.825628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 13:29:52.825628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 13:29:52.825628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 13:29:52.825628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 13:29:52.825628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception Jul 12 13:29:52.829546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 13:29:52.829546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 13:29:52.829546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 13:29:52.829546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 13:29:52.829546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 13:29:52.829546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 13:29:52.829546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 13:29:52.829546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application conn.query(q) Jul 12 13:29:52.829546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 13:29:52.829546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 13:29:52.829546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 13:29:52.829546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application result.read() Jul 12 13:29:52.829546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 13:29:52.829546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 13:29:52.829546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 13:29:52.829546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 13:29:52.831707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 13:29:52.831707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 13:29:52.831707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 13:29:52.831707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 13:29:52.831707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 13:29:52.831707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 12 13:29:52.831707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c839bc9c48b3433d8f329d1737f0d4fa', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 12 13:29:52.831707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 13:29:52.831707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:52.831707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:52.831707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:52.831707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:52.831707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:52.831707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:52.831707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:52.833813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:52.833813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:52.833813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:52.833813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:52.833813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:52.833813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:52.833813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:52.833813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 12 13:29:52.833813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 12 13:29:52.833813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:52.833813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:52.833813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:52.833813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:52.833813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:52.833813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:52.833813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 12 13:29:52.833813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 12 13:29:52.836888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 12 13:29:52.836888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 12 13:29:52.836888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 13:29:52.836888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 13:29:52.836888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 12 13:29:52.836888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:52.836888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 476/975] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Jul 12 13:29:52 2022] POST /identity/v3/users => generated 160 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:52.836888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6a0c5-eba1-40bf-89c6-ca048282a699 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:52.837896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19a7cc-5517-46cf-8903-15bd3942b855 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:52.873997 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e6fc3-7a3b-4ecb-8dd1-1653a7c19cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:52.874873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e6fc3-7a3b-4ecb-8dd1-1653a7c19cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:52.875414 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e6fc3-7a3b-4ecb-8dd1-1653a7c19cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:52.875844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e6fc3-7a3b-4ecb-8dd1-1653a7c19cd8 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:52.876985 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e6fc3-7a3b-4ecb-8dd1-1653a7c19cd8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:52.877395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e6fc3-7a3b-4ecb-8dd1-1653a7c19cd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:52.878980 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e6fc3-7a3b-4ecb-8dd1-1653a7c19cd8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:52.885593 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 500/976] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jul 12 13:29:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:52.890886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e6fc3-7a3b-4ecb-8dd1-1653a7c19cd8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:52.954105 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6a0c5-eba1-40bf-89c6-ca048282a699 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:52.969438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:52.998812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d00ec-49cc-47ae-8c28-37167b6b9282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:52.999677 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d00ec-49cc-47ae-8c28-37167b6b9282 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:53.000733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d00ec-49cc-47ae-8c28-37167b6b9282 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:53.001315 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d00ec-49cc-47ae-8c28-37167b6b9282 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:53.008670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e6fc3-7a3b-4ecb-8dd1-1653a7c19cd8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:53.011562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d00ec-49cc-47ae-8c28-37167b6b9282 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:53.013631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d00ec-49cc-47ae-8c28-37167b6b9282 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:53.036055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-4e3d00ec-49cc-47ae-8c28-37167b6b9282 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 13:29:53.036055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:53.036055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:53.036055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:53.036055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:53.036055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:53.036055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:53.036055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:53.036055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:53.036055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:53.036055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:53.036055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:53.036055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:29:53.036055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:29:53.036055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:29:53.036055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:53.037948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:53.040056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:53.040056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:53.040056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:29:53.040056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:29:53.040056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:29:53.040056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:29:53.040056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:29:53.040056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:29:53.040056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 13:29:53.040056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:53.042319 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 477/977] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:29:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:53.044037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d00ec-49cc-47ae-8c28-37167b6b9282 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:53.082778 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36aedf-1fe4-4b09-9e35-350e6611d882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:53.083521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36aedf-1fe4-4b09-9e35-350e6611d882 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:53.084076 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36aedf-1fe4-4b09-9e35-350e6611d882 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:53.084368 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36aedf-1fe4-4b09-9e35-350e6611d882 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:53.085729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36aedf-1fe4-4b09-9e35-350e6611d882 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:53.087445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36aedf-1fe4-4b09-9e35-350e6611d882 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:53.094081 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:53.119121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 501/978] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jul 12 13:29:52 2022] POST /identity/v3/endpoints => generated 349 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:53.121549 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33a975-bc71-4f0e-9976-802885ad599c None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:53.122015 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33a975-bc71-4f0e-9976-802885ad599c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:53.122447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33a975-bc71-4f0e-9976-802885ad599c None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:53.124547 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 502/979] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:53.130735 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a66b9-5c3c-43d9-a15b-201c51cb3bcb None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:53.131376 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a66b9-5c3c-43d9-a15b-201c51cb3bcb None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:53.131813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a66b9-5c3c-43d9-a15b-201c51cb3bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:53.163324 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9b3a66b9-5c3c-43d9-a15b-201c51cb3bcb None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:53.165267 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9b3a66b9-5c3c-43d9-a15b-201c51cb3bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:53.191471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d00ec-49cc-47ae-8c28-37167b6b9282 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:53.253213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0a416-f4a2-4a81-8cad-183cc7ea7d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:53.253905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0a416-f4a2-4a81-8cad-183cc7ea7d03 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:53.254220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0a416-f4a2-4a81-8cad-183cc7ea7d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:53.254497 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0a416-f4a2-4a81-8cad-183cc7ea7d03 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:53.254678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 503/980] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:29:53.255324 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0a416-f4a2-4a81-8cad-183cc7ea7d03 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:53.255665 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0a416-f4a2-4a81-8cad-183cc7ea7d03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:53.257139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0a416-f4a2-4a81-8cad-183cc7ea7d03 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:53.283112 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a66b9-5c3c-43d9-a15b-201c51cb3bcb None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:53.298791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 478/981] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jul 12 13:29:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:53.349862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:53.403789 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a66b9-5c3c-43d9-a15b-201c51cb3bcb None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:53.444982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612ed4f-06e4-4925-8973-2ff45861c894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:53.445707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612ed4f-06e4-4925-8973-2ff45861c894 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:53.446048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612ed4f-06e4-4925-8973-2ff45861c894 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:53.446336 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612ed4f-06e4-4925-8973-2ff45861c894 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:53.447078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612ed4f-06e4-4925-8973-2ff45861c894 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:53.448532 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612ed4f-06e4-4925-8973-2ff45861c894 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:53.464629 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 504/982] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jul 12 13:29:53 2022] POST /identity/v3/services => generated 242 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:54.753226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ccef4-216c-49dd-8b62-b267a17a5a6b None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:54.753796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ccef4-216c-49dd-8b62-b267a17a5a6b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:54.753796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ccef4-216c-49dd-8b62-b267a17a5a6b None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:54.755321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 479/983] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:54.760816 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3253d2-f0aa-464d-a17f-c3bccbc77090 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:54.761224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3253d2-f0aa-464d-a17f-c3bccbc77090 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:54.761384 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3253d2-f0aa-464d-a17f-c3bccbc77090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:54.778056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3b3253d2-f0aa-464d-a17f-c3bccbc77090 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:54.779258 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3b3253d2-f0aa-464d-a17f-c3bccbc77090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:54.861215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 505/984] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:54.883472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ccef4-216c-49dd-8b62-b267a17a5a6b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:54.975190 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:54.987652 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ccef4-216c-49dd-8b62-b267a17a5a6b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:55.024697 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22aab5-59bf-42bf-af08-15842f7cb600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:55.025423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22aab5-59bf-42bf-af08-15842f7cb600 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:55.025701 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22aab5-59bf-42bf-af08-15842f7cb600 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:55.025995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22aab5-59bf-42bf-af08-15842f7cb600 None admin] PATH_INFO: `/v3/services/image` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:55.026656 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22aab5-59bf-42bf-af08-15842f7cb600 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:55.028053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22aab5-59bf-42bf-af08-15842f7cb600 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:55.040293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-bc22aab5-59bf-42bf-af08-15842f7cb600 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:29:55.040293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:55.040293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:55.040293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:55.040293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:55.040293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:55.040293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:55.040293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:55.040293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:29:55.040293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:55.040293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:55.040293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:55.040293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:55.040293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:55.040293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:55.040293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:55.040293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:55.041828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:55.041828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:55.041828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:55.041828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:55.041828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:55.041828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:55.041828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:55.041828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:55.041828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:55.041828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:55.041828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:55.041828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:55.041828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:55.041828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:55.041828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:55.041828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:55.047630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:55.047630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:55.047630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:55.047630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:55.047630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:55.047630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:55.047630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:55.047630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:55.047630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:55.047630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:55.047630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:55.047630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:55.047630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:29:55.047630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:55.047630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 480/985] 10.222.0.52 () {56 vars in 1105 bytes} [Tue Jul 12 13:29:54 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:55.048430 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3253d2-f0aa-464d-a17f-c3bccbc77090 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:55.097693 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:55.162938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3253d2-f0aa-464d-a17f-c3bccbc77090 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:55.197613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef761e2-a326-4b62-9348-f0d9d3eead19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:55.198304 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef761e2-a326-4b62-9348-f0d9d3eead19 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:55.198590 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef761e2-a326-4b62-9348-f0d9d3eead19 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:55.198867 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef761e2-a326-4b62-9348-f0d9d3eead19 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:55.199569 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef761e2-a326-4b62-9348-f0d9d3eead19 None admin] RBAC: Adding query filter params (name=image) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:55.199876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef761e2-a326-4b62-9348-f0d9d3eead19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:55.201256 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef761e2-a326-4b62-9348-f0d9d3eead19 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:55.211305 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 506/986] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:55.212425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22aab5-59bf-42bf-af08-15842f7cb600 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:55.268952 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13caf5b9-943a-4e50-833f-7e7d5f479f77 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:55.269286 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13caf5b9-943a-4e50-833f-7e7d5f479f77 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:55.269514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13caf5b9-943a-4e50-833f-7e7d5f479f77 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:55.270985 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 507/987] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:29:55.277819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b786e-199e-4475-a826-492d54ce779c None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:55.278077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b786e-199e-4475-a826-492d54ce779c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:55.278361 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b786e-199e-4475-a826-492d54ce779c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:55.296495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-769b786e-199e-4475-a826-492d54ce779c None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:55.297699 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-769b786e-199e-4475-a826-492d54ce779c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:55.322397 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22aab5-59bf-42bf-af08-15842f7cb600 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:55.359151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e703cd5-e072-40dc-bca3-b07642e47266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:55.359818 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e703cd5-e072-40dc-bca3-b07642e47266 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:55.360208 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e703cd5-e072-40dc-bca3-b07642e47266 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:55.360368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e703cd5-e072-40dc-bca3-b07642e47266 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:55.361068 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e703cd5-e072-40dc-bca3-b07642e47266 None admin] RBAC: Adding query filter params (type=image) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:55.361385 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e703cd5-e072-40dc-bca3-b07642e47266 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:55.362759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e703cd5-e072-40dc-bca3-b07642e47266 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:55.366063 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 508/988] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:29:55.367956 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 481/989] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:29:55 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:55.389183 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b786e-199e-4475-a826-492d54ce779c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:55.427483 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:55.501197 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b786e-199e-4475-a826-492d54ce779c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:55.508051 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0c4e2-6ec2-4014-bec5-03da37c1a3df None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:55.508742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0c4e2-6ec2-4014-bec5-03da37c1a3df None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:55.509617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0c4e2-6ec2-4014-bec5-03da37c1a3df None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:55.513390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 482/990] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:55.518417 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a70f8a-9434-45a2-ae64-ee19a2b99e15 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:55.518691 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a70f8a-9434-45a2-ae64-ee19a2b99e15 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:55.518984 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a70f8a-9434-45a2-ae64-ee19a2b99e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:55.542977 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d7a70f8a-9434-45a2-ae64-ee19a2b99e15 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:55.544832 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2b340-d4fb-42ec-88fc-972b279af8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:55.544963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d7a70f8a-9434-45a2-ae64-ee19a2b99e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:55.545655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2b340-d4fb-42ec-88fc-972b279af8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:55.545943 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2b340-d4fb-42ec-88fc-972b279af8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:55.546253 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2b340-d4fb-42ec-88fc-972b279af8c3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:55.546986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2b340-d4fb-42ec-88fc-972b279af8c3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:55.548300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2b340-d4fb-42ec-88fc-972b279af8c3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:55.560994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-5af2b340-d4fb-42ec-88fc-972b279af8c3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:29:55.560994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:55.560994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:55.560994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:55.560994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:55.560994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:55.560994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:55.560994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:55.560994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:29:55.560994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:55.560994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:55.560994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:55.560994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:55.560994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:55.560994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:55.560994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:55.560994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:55.562356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:55.562356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:55.562356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:55.562356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:55.562356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:55.562356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:55.562356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:55.562356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:55.562356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:55.562356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:55.562356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:55.562356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:55.562356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:55.562356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:55.562356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:55.562356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:55.563234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:55.563234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:55.563234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:55.563234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:55.563234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:55.563234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:55.563234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:55.563234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:55.563234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:55.563234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:55.563234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:55.563234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:55.563234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:29:55.563234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:55.563234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 509/991] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jul 12 13:29:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:55.569331 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2b340-d4fb-42ec-88fc-972b279af8c3 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:55.569481 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:55.630680 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 483/992] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:29:55.654277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a70f8a-9434-45a2-ae64-ee19a2b99e15 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:55.672345 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:55.677292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2b340-d4fb-42ec-88fc-972b279af8c3 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:55.717932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a840da-cab6-4865-9a21-d171424977d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:55.718729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a840da-cab6-4865-9a21-d171424977d6 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:55.718915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a840da-cab6-4865-9a21-d171424977d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:55.719193 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a840da-cab6-4865-9a21-d171424977d6 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:55.719909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a840da-cab6-4865-9a21-d171424977d6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:55.720220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a840da-cab6-4865-9a21-d171424977d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:55.721605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a840da-cab6-4865-9a21-d171424977d6 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:55.726979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 510/993] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jul 12 13:29:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:55.732251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a840da-cab6-4865-9a21-d171424977d6 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:55.769736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a70f8a-9434-45a2-ae64-ee19a2b99e15 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:55.806506 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810a8b9-481f-4469-9f3e-1f8cb9417a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:55.807297 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810a8b9-481f-4469-9f3e-1f8cb9417a37 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:55.807553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810a8b9-481f-4469-9f3e-1f8cb9417a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:55.807829 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810a8b9-481f-4469-9f3e-1f8cb9417a37 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:55.808946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810a8b9-481f-4469-9f3e-1f8cb9417a37 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:55.810494 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810a8b9-481f-4469-9f3e-1f8cb9417a37 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:55.824085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-3810a8b9-481f-4469-9f3e-1f8cb9417a37 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:29:55.824085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:55.824085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:55.824085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:55.824085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:55.824085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:55.824085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:55.824085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:55.824085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:29:55.824085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:55.824085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:55.824085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:55.824085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:55.824085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:55.824085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:55.824085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:55.824085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:55.825138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:55.825138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:55.825138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:55.825138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:55.825138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:55.825138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:55.825138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:55.825138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:55.825138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:55.825138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:55.825138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:55.825138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:55.825138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:55.825138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:55.825138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:55.825138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:55.825996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:55.825996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:55.825996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:55.825996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:55.825996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:55.825996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:55.825996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:55.825996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:55.825996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:55.825996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:55.825996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:55.825996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:55.825996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:29:55.825996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:55.825996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 484/994] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:55.833021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810a8b9-481f-4469-9f3e-1f8cb9417a37 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:55.839431 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a840da-cab6-4865-9a21-d171424977d6 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:55.853386 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:55.880190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d9262-9e25-445f-9934-956db2821f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:55.880980 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d9262-9e25-445f-9934-956db2821f61 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:55.881340 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d9262-9e25-445f-9934-956db2821f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:55.881692 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d9262-9e25-445f-9934-956db2821f61 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:55.882565 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d9262-9e25-445f-9934-956db2821f61 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:55.882976 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d9262-9e25-445f-9934-956db2821f61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:55.884465 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d9262-9e25-445f-9934-956db2821f61 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:55.894942 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 511/995] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jul 12 13:29:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:55.895956 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d9262-9e25-445f-9934-956db2821f61 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:55.925206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:55.956685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810a8b9-481f-4469-9f3e-1f8cb9417a37 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:55.994100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6095f-2b21-417d-9328-995e5e960626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:55.995441 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6095f-2b21-417d-9328-995e5e960626 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:55.995728 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6095f-2b21-417d-9328-995e5e960626 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:55.996004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6095f-2b21-417d-9328-995e5e960626 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:55.996713 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e6095f-2b21-417d-9328-995e5e960626 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:55.997022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e6095f-2b21-417d-9328-995e5e960626 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:55.998444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e6095f-2b21-417d-9328-995e5e960626 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:56.003598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d9262-9e25-445f-9934-956db2821f61 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:56.004267 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 485/996] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jul 12 13:29:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:56.009813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6095f-2b21-417d-9328-995e5e960626 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:56.044585 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2ff53-8d7f-4c8a-b1a6-df1aa4d82921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:56.045332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2ff53-8d7f-4c8a-b1a6-df1aa4d82921 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:56.045690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2ff53-8d7f-4c8a-b1a6-df1aa4d82921 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:56.046101 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2ff53-8d7f-4c8a-b1a6-df1aa4d82921 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:56.046877 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2ff53-8d7f-4c8a-b1a6-df1aa4d82921 None admin] RBAC: Adding query filter params (interface=admin, service_id=ac4a0f831dfc4cf7a4d650d7dfffb590, region_id=RegionOne) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:56.047192 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2ff53-8d7f-4c8a-b1a6-df1aa4d82921 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:56.048579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2ff53-8d7f-4c8a-b1a6-df1aa4d82921 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:56.059007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 512/997] 10.222.0.52 () {56 vars in 1249 bytes} [Tue Jul 12 13:29:55 2022] GET /identity/v3/endpoints?service_id=ac4a0f831dfc4cf7a4d650d7dfffb590&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:56.060413 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2ff53-8d7f-4c8a-b1a6-df1aa4d82921 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:56.092705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:56.117605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6095f-2b21-417d-9328-995e5e960626 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:56.153046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e346ca3-7b37-4928-8675-f95784ffb298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:56.153856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e346ca3-7b37-4928-8675-f95784ffb298 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:56.154165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e346ca3-7b37-4928-8675-f95784ffb298 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:56.154439 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e346ca3-7b37-4928-8675-f95784ffb298 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:56.155143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e346ca3-7b37-4928-8675-f95784ffb298 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:56.155449 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e346ca3-7b37-4928-8675-f95784ffb298 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:56.156818 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e346ca3-7b37-4928-8675-f95784ffb298 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:56.160436 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:56.161727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 486/998] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jul 12 13:29:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:56.167739 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e346ca3-7b37-4928-8675-f95784ffb298 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:56.171890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2ff53-8d7f-4c8a-b1a6-df1aa4d82921 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:56.211093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a343f32-1897-481b-a07c-cf87239f4c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:56.211855 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a343f32-1897-481b-a07c-cf87239f4c9e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:56.212211 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a343f32-1897-481b-a07c-cf87239f4c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:56.212838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a343f32-1897-481b-a07c-cf87239f4c9e None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:56.213628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a343f32-1897-481b-a07c-cf87239f4c9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:56.215123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a343f32-1897-481b-a07c-cf87239f4c9e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:56.221759 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 513/999] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jul 12 13:29:56 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:56.273982 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e346ca3-7b37-4928-8675-f95784ffb298 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:56.312167 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09c8a4-afcd-4374-a180-928afdc3606b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:56.312746 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09c8a4-afcd-4374-a180-928afdc3606b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:56.313029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09c8a4-afcd-4374-a180-928afdc3606b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:56.313304 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09c8a4-afcd-4374-a180-928afdc3606b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:56.314134 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09c8a4-afcd-4374-a180-928afdc3606b None admin] RBAC: Adding query filter params (interface=public, service_id=685038ba5d1343a5bb51c14268f722cc, region_id=RegionOne) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:56.314446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09c8a4-afcd-4374-a180-928afdc3606b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:56.315836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09c8a4-afcd-4374-a180-928afdc3606b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:56.321819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 487/1000] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Jul 12 13:29:56 2022] GET /identity/v3/endpoints?service_id=685038ba5d1343a5bb51c14268f722cc&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:56.327260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a343f32-1897-481b-a07c-cf87239f4c9e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:56.436769 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a343f32-1897-481b-a07c-cf87239f4c9e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:56.471152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376493de-cc69-4a23-8cfe-bf723568b3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:56.471973 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376493de-cc69-4a23-8cfe-bf723568b3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:56.472388 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376493de-cc69-4a23-8cfe-bf723568b3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:56.472766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376493de-cc69-4a23-8cfe-bf723568b3b3 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:56.473559 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376493de-cc69-4a23-8cfe-bf723568b3b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:56.475085 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376493de-cc69-4a23-8cfe-bf723568b3b3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:56.482727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 514/1001] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jul 12 13:29:56 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:57.408005 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c55b0-9a6c-4a41-bb04-36c967feb050 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:57.408530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c55b0-9a6c-4a41-bb04-36c967feb050 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:57.408638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c55b0-9a6c-4a41-bb04-36c967feb050 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:57.410088 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 488/1002] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:57.415179 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234304b3-9cd8-47a1-a4f1-9565fe4cd9fe None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:57.415583 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234304b3-9cd8-47a1-a4f1-9565fe4cd9fe None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:57.415769 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234304b3-9cd8-47a1-a4f1-9565fe4cd9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:57.432848 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-234304b3-9cd8-47a1-a4f1-9565fe4cd9fe None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:57.434063 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-234304b3-9cd8-47a1-a4f1-9565fe4cd9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:57.447657 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:57.504314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 515/1003] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:57.526781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c55b0-9a6c-4a41-bb04-36c967feb050 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:57.531662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:57.637576 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c55b0-9a6c-4a41-bb04-36c967feb050 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:57.674286 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268ecee-acc9-4d91-83d9-e4f83dff6fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:57.674875 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268ecee-acc9-4d91-83d9-e4f83dff6fc1 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:57.675153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268ecee-acc9-4d91-83d9-e4f83dff6fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:57.675423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268ecee-acc9-4d91-83d9-e4f83dff6fc1 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:57.676158 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268ecee-acc9-4d91-83d9-e4f83dff6fc1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:57.678338 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268ecee-acc9-4d91-83d9-e4f83dff6fc1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:57.695509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 489/1004] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jul 12 13:29:57 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:58.285012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b9665-62cc-491c-b925-141ff8c89895 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:58.285405 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b9665-62cc-491c-b925-141ff8c89895 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:58.285610 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b9665-62cc-491c-b925-141ff8c89895 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:58.287006 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 516/1005] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:58.292075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907db6e0-9c95-4b10-a511-7f5f4d9f3cb3 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:58.292368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907db6e0-9c95-4b10-a511-7f5f4d9f3cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:58.292649 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907db6e0-9c95-4b10-a511-7f5f4d9f3cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:58.309664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-907db6e0-9c95-4b10-a511-7f5f4d9f3cb3 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:58.310959 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-907db6e0-9c95-4b10-a511-7f5f4d9f3cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:58.371267 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:58.388367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 490/1006] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:58 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:58.410595 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b9665-62cc-491c-b925-141ff8c89895 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:58.516514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b9665-62cc-491c-b925-141ff8c89895 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:58.527494 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8270889-3301-4c30-b8cd-25a24b9750e5 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:58.527965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8270889-3301-4c30-b8cd-25a24b9750e5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:58.528055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8270889-3301-4c30-b8cd-25a24b9750e5 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:58.529634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 491/1007] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:29:58.534366 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138f8be-bcfc-4d6e-8cf4-f2e642a3ca27 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:58.534657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138f8be-bcfc-4d6e-8cf4-f2e642a3ca27 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:58.534942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138f8be-bcfc-4d6e-8cf4-f2e642a3ca27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:58.552518 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7138f8be-bcfc-4d6e-8cf4-f2e642a3ca27 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:58.553635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7138f8be-bcfc-4d6e-8cf4-f2e642a3ca27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:58.554765 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e3459-a18c-4cd4-828f-2ef8a641fef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:58.555577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e3459-a18c-4cd4-828f-2ef8a641fef8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:58.555864 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e3459-a18c-4cd4-828f-2ef8a641fef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:58.556138 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e3459-a18c-4cd4-828f-2ef8a641fef8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:58.557653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e3459-a18c-4cd4-828f-2ef8a641fef8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:58.559337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e3459-a18c-4cd4-828f-2ef8a641fef8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:58.572557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-914e3459-a18c-4cd4-828f-2ef8a641fef8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:29:58.572557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:58.572557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:58.572557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:58.572557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:58.572557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:58.572557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:58.572557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:58.572557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:29:58.572557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:58.572557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:58.572557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:58.572557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:58.572557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:58.572557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:58.572557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:58.572557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:58.573542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:58.573542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:58.573542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:58.573542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:58.573542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:58.573542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:58.573542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:58.573542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:58.573542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:58.573542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:58.573542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:58.573542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:58.573542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:58.573542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:58.573542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:58.573542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:58.574496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:58.574496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:58.574496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:58.574496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:58.574496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:58.574496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:58.574496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:58.574496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:58.574496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:58.574496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:58.574496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:58.574496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:58.574496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:29:58.574496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:29:58.574496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 517/1008] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jul 12 13:29:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:29:58.579742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e3459-a18c-4cd4-828f-2ef8a641fef8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:58.596512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:58.630365 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 492/1009] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:58 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:29:58.653179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138f8be-bcfc-4d6e-8cf4-f2e642a3ca27 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:58.696539 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e3459-a18c-4cd4-828f-2ef8a641fef8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:58.730839 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67787903-b7aa-4ffe-b6ad-155b82671ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:58.731505 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67787903-b7aa-4ffe-b6ad-155b82671ccf None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:58.731815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67787903-b7aa-4ffe-b6ad-155b82671ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:58.732072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67787903-b7aa-4ffe-b6ad-155b82671ccf None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:58.732781 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67787903-b7aa-4ffe-b6ad-155b82671ccf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:58.733093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67787903-b7aa-4ffe-b6ad-155b82671ccf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:58.734524 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67787903-b7aa-4ffe-b6ad-155b82671ccf None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:58.739530 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 518/1010] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jul 12 13:29:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:58.746020 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67787903-b7aa-4ffe-b6ad-155b82671ccf None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:58.764416 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138f8be-bcfc-4d6e-8cf4-f2e642a3ca27 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:58.786422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:58.803923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63c907-a1e9-4599-998f-5b0a408481c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:58.804657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63c907-a1e9-4599-998f-5b0a408481c1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:58.804944 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63c907-a1e9-4599-998f-5b0a408481c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:58.805220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63c907-a1e9-4599-998f-5b0a408481c1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:58.805880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63c907-a1e9-4599-998f-5b0a408481c1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:58.807954 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63c907-a1e9-4599-998f-5b0a408481c1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:58.820264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-3b63c907-a1e9-4599-998f-5b0a408481c1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:29:58.820264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:58.820264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:29:58.820264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:29:58.820264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:29:58.820264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:29:58.820264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:29:58.820264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:58.820264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:29:58.820264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:58.820264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:29:58.820264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:58.820264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:29:58.820264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:29:58.820264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:29:58.820264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:29:58.820264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:29:58.821561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:29:58.821561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:29:58.821561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:29:58.821561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:29:58.821561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:29:58.821561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:29:58.821561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:29:58.821561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:29:58.821561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:29:58.821561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:29:58.821561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:29:58.821561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:29:58.821561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:29:58.821561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:29:58.821561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:29:58.821561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:29:58.822522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:29:58.822522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:29:58.822522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:29:58.822522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:29:58.822522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:29:58.822522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:29:58.822522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:29:58.822522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:29:58.822522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:29:58.822522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:29:58.822522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:29:58.822522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:29:58.822522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:29:58.822522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:29:58.822522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 493/1011] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:29:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:29:58.828129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63c907-a1e9-4599-998f-5b0a408481c1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:58.840529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:58.871060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67787903-b7aa-4ffe-b6ad-155b82671ccf None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:58.907305 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dff3d4-a2a6-4a2f-9a35-41cd45e77c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:58.908301 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dff3d4-a2a6-4a2f-9a35-41cd45e77c07 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:58.908880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dff3d4-a2a6-4a2f-9a35-41cd45e77c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:58.909372 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dff3d4-a2a6-4a2f-9a35-41cd45e77c07 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:58.910345 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dff3d4-a2a6-4a2f-9a35-41cd45e77c07 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:58.910720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dff3d4-a2a6-4a2f-9a35-41cd45e77c07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:58.912104 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dff3d4-a2a6-4a2f-9a35-41cd45e77c07 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:58.922457 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 519/1012] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jul 12 13:29:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:58.923419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dff3d4-a2a6-4a2f-9a35-41cd45e77c07 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:58.944755 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63c907-a1e9-4599-998f-5b0a408481c1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:58.987857 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd18e67-4ab4-4df9-9baf-fec32105977c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:58.988897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd18e67-4ab4-4df9-9baf-fec32105977c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:58.989320 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd18e67-4ab4-4df9-9baf-fec32105977c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:58.989728 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd18e67-4ab4-4df9-9baf-fec32105977c None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:58.990475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd18e67-4ab4-4df9-9baf-fec32105977c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:58.990931 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd18e67-4ab4-4df9-9baf-fec32105977c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:58.992892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd18e67-4ab4-4df9-9baf-fec32105977c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:59.000157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 494/1013] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jul 12 13:29:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:59.005338 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd18e67-4ab4-4df9-9baf-fec32105977c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:59.033685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dff3d4-a2a6-4a2f-9a35-41cd45e77c07 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:59.061434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:59.070793 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16098d8-91b6-491e-b232-e5cc4412f382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:59.071449 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16098d8-91b6-491e-b232-e5cc4412f382 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:59.071732 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16098d8-91b6-491e-b232-e5cc4412f382 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:59.072012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16098d8-91b6-491e-b232-e5cc4412f382 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:59.072776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16098d8-91b6-491e-b232-e5cc4412f382 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:59.074180 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16098d8-91b6-491e-b232-e5cc4412f382 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:59.084550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:59.102181 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 520/1014] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jul 12 13:29:58 2022] POST /identity/v3/endpoints => generated 325 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:29:59.120677 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd18e67-4ab4-4df9-9baf-fec32105977c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:59.173233 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e6130-7730-4acd-a7de-ec312c93f94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:59.173931 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e6130-7730-4acd-a7de-ec312c93f94d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:59.174268 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e6130-7730-4acd-a7de-ec312c93f94d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:59.174553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e6130-7730-4acd-a7de-ec312c93f94d None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:59.175289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e6130-7730-4acd-a7de-ec312c93f94d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:59.175604 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e6130-7730-4acd-a7de-ec312c93f94d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:59.177050 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e6130-7730-4acd-a7de-ec312c93f94d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:59.188109 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 495/1015] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jul 12 13:29:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:59.191093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16098d8-91b6-491e-b232-e5cc4412f382 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:59.305627 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16098d8-91b6-491e-b232-e5cc4412f382 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:59.347246 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12206587-556c-4f32-90cc-2ffb31d8a9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:59.347835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12206587-556c-4f32-90cc-2ffb31d8a9f0 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:59.348127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12206587-556c-4f32-90cc-2ffb31d8a9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:59.348405 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12206587-556c-4f32-90cc-2ffb31d8a9f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:59.349151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12206587-556c-4f32-90cc-2ffb31d8a9f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:59.350598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12206587-556c-4f32-90cc-2ffb31d8a9f0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:59.377592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 521/1016] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jul 12 13:29:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:59.768654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0bf27-aac2-42bf-8771-197eefd27376 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:59.769198 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0bf27-aac2-42bf-8771-197eefd27376 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:59.769198 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0bf27-aac2-42bf-8771-197eefd27376 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:59.770800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 496/1017] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:29:59.775879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546156d7-9ef4-40e9-bc51-4996b8423b08 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:59.776272 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546156d7-9ef4-40e9-bc51-4996b8423b08 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:59.776454 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546156d7-9ef4-40e9-bc51-4996b8423b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:59.793374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-546156d7-9ef4-40e9-bc51-4996b8423b08 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:59.794641 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-546156d7-9ef4-40e9-bc51-4996b8423b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:59.850594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:59.877727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 522/1018] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:29:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:29:59.900503 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0bf27-aac2-42bf-8771-197eefd27376 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:00.005186 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0bf27-aac2-42bf-8771-197eefd27376 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:00.016300 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:00.043277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c61b2d-819b-4d45-9674-3ea071d152c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:00.043986 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c61b2d-819b-4d45-9674-3ea071d152c9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:00.044273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c61b2d-819b-4d45-9674-3ea071d152c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:00.044552 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c61b2d-819b-4d45-9674-3ea071d152c9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:00.045205 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c61b2d-819b-4d45-9674-3ea071d152c9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:00.046606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c61b2d-819b-4d45-9674-3ea071d152c9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:00.058781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-a5c61b2d-819b-4d45-9674-3ea071d152c9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:30:00.058781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:00.058781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:30:00.058781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:30:00.058781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:30:00.058781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:30:00.058781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:30:00.058781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:30:00.058781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:30:00.058781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:00.058781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:30:00.058781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:00.058781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:00.058781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:00.058781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:00.058781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:00.058781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:00.059765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:00.059765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:00.059765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:00.059765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:00.059765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:00.059765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:00.059765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:30:00.059765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:30:00.059765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:30:00.059765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:30:00.059765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:00.059765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:00.059765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:00.059765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:00.059765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:00.059765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:00.060643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:00.060643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:00.060643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:00.060643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:00.060643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:00.060643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:00.060643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:00.060643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:00.060643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:00.060643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:00.060643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:30:00.060643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:30:00.060643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:30:00.060643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:00.064932 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 497/1019] 10.222.0.52 () {56 vars in 1105 bytes} [Tue Jul 12 13:29:59 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:00.065926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546156d7-9ef4-40e9-bc51-4996b8423b08 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:00.168319 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546156d7-9ef4-40e9-bc51-4996b8423b08 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:00.203139 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af361fdf-07df-48c4-a856-0e2e913b80dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:00.203810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af361fdf-07df-48c4-a856-0e2e913b80dd None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:00.204096 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af361fdf-07df-48c4-a856-0e2e913b80dd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:00.204375 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af361fdf-07df-48c4-a856-0e2e913b80dd None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:00.205081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af361fdf-07df-48c4-a856-0e2e913b80dd None admin] RBAC: Adding query filter params (name=image) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:00.205393 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af361fdf-07df-48c4-a856-0e2e913b80dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:00.206821 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af361fdf-07df-48c4-a856-0e2e913b80dd None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:00.216472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 523/1020] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:30:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:00.217637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c61b2d-819b-4d45-9674-3ea071d152c9 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:00.321788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c61b2d-819b-4d45-9674-3ea071d152c9 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:00.355251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75f31a-d401-4bfd-a9c9-f4b2e7fa6be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:00.355864 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75f31a-d401-4bfd-a9c9-f4b2e7fa6be3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:00.356148 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75f31a-d401-4bfd-a9c9-f4b2e7fa6be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:00.356422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75f31a-d401-4bfd-a9c9-f4b2e7fa6be3 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:00.357121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75f31a-d401-4bfd-a9c9-f4b2e7fa6be3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:00.357583 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75f31a-d401-4bfd-a9c9-f4b2e7fa6be3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:00.358837 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75f31a-d401-4bfd-a9c9-f4b2e7fa6be3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:00.368605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 498/1021] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:30:00 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:00.370420 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af361fdf-07df-48c4-a856-0e2e913b80dd None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:00.418440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:00.472435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af361fdf-07df-48c4-a856-0e2e913b80dd None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:00.507573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebb412-5569-46e7-8cea-8f181480965c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:00.508234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebb412-5569-46e7-8cea-8f181480965c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:00.508520 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebb412-5569-46e7-8cea-8f181480965c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:00.508798 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebb412-5569-46e7-8cea-8f181480965c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:00.509583 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebb412-5569-46e7-8cea-8f181480965c None admin] RBAC: Adding query filter params (interface=public, service_id=cee3136a3e5846f6ae4ab62a64565820, region_id=RegionOne) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:00.509929 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebb412-5569-46e7-8cea-8f181480965c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:00.511354 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebb412-5569-46e7-8cea-8f181480965c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:00.516837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 524/1022] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Jul 12 13:30:00 2022] GET /identity/v3/endpoints?service_id=cee3136a3e5846f6ae4ab62a64565820&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:00.522241 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75f31a-d401-4bfd-a9c9-f4b2e7fa6be3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:00.562524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:00.636283 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75f31a-d401-4bfd-a9c9-f4b2e7fa6be3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:00.672118 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3d1bf-dde0-4174-9001-45319c9b0dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:00.672782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3d1bf-dde0-4174-9001-45319c9b0dde None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:00.673064 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3d1bf-dde0-4174-9001-45319c9b0dde None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:00.674164 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3d1bf-dde0-4174-9001-45319c9b0dde None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:00.674164 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3d1bf-dde0-4174-9001-45319c9b0dde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:00.675392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3d1bf-dde0-4174-9001-45319c9b0dde None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:00.693987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 499/1023] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jul 12 13:30:00 2022] GET /identity/v3/services => generated 1957 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:30:01.211445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d9df2-9ee0-4a6b-a26f-ee0cca94a2a3 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:01.211967 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d9df2-9ee0-4a6b-a26f-ee0cca94a2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:01.212046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d9df2-9ee0-4a6b-a26f-ee0cca94a2a3 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:01.213570 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 525/1024] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:01.218630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274c1db-0ddd-4968-85c9-5ddbe579b5f9 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:01.219052 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274c1db-0ddd-4968-85c9-5ddbe579b5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:01.219229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274c1db-0ddd-4968-85c9-5ddbe579b5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:01.235818 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-2274c1db-0ddd-4968-85c9-5ddbe579b5f9 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:01.237014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2274c1db-0ddd-4968-85c9-5ddbe579b5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:01.307351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 500/1025] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:01.330148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d9df2-9ee0-4a6b-a26f-ee0cca94a2a3 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:01.434814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d9df2-9ee0-4a6b-a26f-ee0cca94a2a3 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:01.445761 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:01.469996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7ff61-9726-43ed-ba77-c5fc338ced32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:01.470727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7ff61-9726-43ed-ba77-c5fc338ced32 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:01.471080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7ff61-9726-43ed-ba77-c5fc338ced32 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:01.471290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7ff61-9726-43ed-ba77-c5fc338ced32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:01.472467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be7ff61-9726-43ed-ba77-c5fc338ced32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:01.473911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be7ff61-9726-43ed-ba77-c5fc338ced32 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:01.475759 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 526/1026] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:30:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:01.481440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274c1db-0ddd-4968-85c9-5ddbe579b5f9 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:01.513774 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:01.581377 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bbe19-51f4-4f66-9a65-276a74adeebb None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:01.582091 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bbe19-51f4-4f66-9a65-276a74adeebb None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:01.582381 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bbe19-51f4-4f66-9a65-276a74adeebb None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:01.583999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 527/1027] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:30:01.588735 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274c1db-0ddd-4968-85c9-5ddbe579b5f9 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:01.590105 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115b330-15c8-4b12-bb7d-a71e03763518 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:01.590553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115b330-15c8-4b12-bb7d-a71e03763518 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:01.591018 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115b330-15c8-4b12-bb7d-a71e03763518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:01.609986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-0115b330-15c8-4b12-bb7d-a71e03763518 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:01.612229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0115b330-15c8-4b12-bb7d-a71e03763518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:01.626458 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc497782-1756-424e-a9af-6b626e601ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:01.627120 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc497782-1756-424e-a9af-6b626e601ae7 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:01.627399 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc497782-1756-424e-a9af-6b626e601ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:01.627678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc497782-1756-424e-a9af-6b626e601ae7 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:01.628388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc497782-1756-424e-a9af-6b626e601ae7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:01.629813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc497782-1756-424e-a9af-6b626e601ae7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:01.654855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 501/1028] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jul 12 13:30:01 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:30:01.695940 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 528/1029] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:30:01.719990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc497782-1756-424e-a9af-6b626e601ae7 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:01.814090 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:01.835011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc497782-1756-424e-a9af-6b626e601ae7 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:01.879463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb08ff7-60cc-431f-91a6-fb8cc0e22b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:01.880287 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb08ff7-60cc-431f-91a6-fb8cc0e22b80 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:01.880574 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb08ff7-60cc-431f-91a6-fb8cc0e22b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:01.880854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb08ff7-60cc-431f-91a6-fb8cc0e22b80 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:01.892065 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb08ff7-60cc-431f-91a6-fb8cc0e22b80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:30:01.900942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-cbb08ff7-60cc-431f-91a6-fb8cc0e22b80 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:30:01.900942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:01.900942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:01.900942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:01.900942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:01.900942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:01.900942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:01.900942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:01.900942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:01.900942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:01.900942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:01.900942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:01.900942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:30:01.900942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:30:01.900942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:30:01.900942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:30:01.901912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:30:01.901912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:30:01.901912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:30:01.901912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:30:01.901912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:01.901912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:01.901912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:01.901912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:01.901912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:01.901912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:01.901912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:01.901912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:01.901912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:01.901912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:01.901912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:01.901912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:01.903231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:01.903231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:01.903231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:01.903231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:01.903231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:30:01.903231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:30:01.903231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:30:01.903231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:30:01.903231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:30:01.903231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:30:01.903231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:30:01.903231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:01.903231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 502/1030] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Jul 12 13:30:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:01.908430 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115b330-15c8-4b12-bb7d-a71e03763518 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:02.009227 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115b330-15c8-4b12-bb7d-a71e03763518 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:02.018729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:02.049067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744c2a0-118c-454e-bcac-820bd17b48c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:02.050246 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744c2a0-118c-454e-bcac-820bd17b48c3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:02.050756 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744c2a0-118c-454e-bcac-820bd17b48c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:02.051254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744c2a0-118c-454e-bcac-820bd17b48c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:02.052648 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744c2a0-118c-454e-bcac-820bd17b48c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:02.053209 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744c2a0-118c-454e-bcac-820bd17b48c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:02.055797 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744c2a0-118c-454e-bcac-820bd17b48c3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:02.066986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 529/1031] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Jul 12 13:30:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:02.072781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb08ff7-60cc-431f-91a6-fb8cc0e22b80 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:02.182652 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb08ff7-60cc-431f-91a6-fb8cc0e22b80 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:02.217461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac636fc-92f8-4f21-a410-83a98b16179c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:02.218280 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac636fc-92f8-4f21-a410-83a98b16179c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:02.218562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac636fc-92f8-4f21-a410-83a98b16179c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:02.218832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac636fc-92f8-4f21-a410-83a98b16179c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:02.230199 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac636fc-92f8-4f21-a410-83a98b16179c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:02.231645 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac636fc-92f8-4f21-a410-83a98b16179c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:02.233443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:02.250519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-cac636fc-92f8-4f21-a410-83a98b16179c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:30:02.250519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:02.250519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:02.250519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:02.250519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:02.250519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:02.250519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:02.250519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:02.250519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:02.250519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:02.250519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:02.250519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:02.250519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:30:02.250519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:30:02.250519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:30:02.251734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:30:02.251734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:02.251734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:02.251734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:02.251734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:02.251734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:02.251734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:02.251734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:02.251734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:02.251734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:02.251734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:02.251734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:02.251734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:02.251734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:02.251734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:02.251734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:02.251734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:02.252665 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:30:02.252665 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:30:02.252665 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:30:02.252665 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:02.254336 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 503/1032] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:02.257785 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744c2a0-118c-454e-bcac-820bd17b48c3 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:02.361228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744c2a0-118c-454e-bcac-820bd17b48c3 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:02.395315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d7359-ff75-481b-9b88-b854234085bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:02.395971 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d7359-ff75-481b-9b88-b854234085bb None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:02.396254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d7359-ff75-481b-9b88-b854234085bb None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:02.396527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d7359-ff75-481b-9b88-b854234085bb None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:02.397240 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d7359-ff75-481b-9b88-b854234085bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:02.397544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d7359-ff75-481b-9b88-b854234085bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:02.398953 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d7359-ff75-481b-9b88-b854234085bb None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:02.416370 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 530/1033] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:02.417561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac636fc-92f8-4f21-a410-83a98b16179c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:02.518549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac636fc-92f8-4f21-a410-83a98b16179c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:02.553221 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6900527-8941-4497-913c-767aa387f5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:02.554061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6900527-8941-4497-913c-767aa387f5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:02.554344 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6900527-8941-4497-913c-767aa387f5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:02.554621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6900527-8941-4497-913c-767aa387f5a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:02.564500 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6900527-8941-4497-913c-767aa387f5a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:02.566010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6900527-8941-4497-913c-767aa387f5a0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:02.580176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-d6900527-8941-4497-913c-767aa387f5a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:30:02.580176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:02.580176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:02.580176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:02.580176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:02.580176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:02.580176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:02.580176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:02.580176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:02.580176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:02.580176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:02.580176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:02.580176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:30:02.580176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:30:02.580176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:30:02.580176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:02.581159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:02.586800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:02.586800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:02.586800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:30:02.586800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:30:02.586800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:30:02.586800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:30:02.586800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:30:02.586800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:30:02.586800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:30:02.586800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:02.586800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 504/1034] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jul 12 13:30:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:02.587423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d7359-ff75-481b-9b88-b854234085bb None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:02.617772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:02.703775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d7359-ff75-481b-9b88-b854234085bb None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:02.740151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb122b8e-6e87-44c9-b058-1c70f228e9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:02.741087 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb122b8e-6e87-44c9-b058-1c70f228e9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:02.741532 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb122b8e-6e87-44c9-b058-1c70f228e9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:02.741953 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb122b8e-6e87-44c9-b058-1c70f228e9e7 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:02.742960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb122b8e-6e87-44c9-b058-1c70f228e9e7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:02.743411 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb122b8e-6e87-44c9-b058-1c70f228e9e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:02.745029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb122b8e-6e87-44c9-b058-1c70f228e9e7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:02.763481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 531/1035] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jul 12 13:30:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:02.769135 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6900527-8941-4497-913c-767aa387f5a0 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:02.788593 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42856ad8-6390-4c5c-af86-80b1c1e4efef None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:02.788890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42856ad8-6390-4c5c-af86-80b1c1e4efef None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:02.789170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42856ad8-6390-4c5c-af86-80b1c1e4efef None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:02.790736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 532/1036] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:30:02.796326 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911232b-89f0-4be3-baa0-03e1fcbd5f79 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:02.796618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911232b-89f0-4be3-baa0-03e1fcbd5f79 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:02.796904 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911232b-89f0-4be3-baa0-03e1fcbd5f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:02.813409 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-6911232b-89f0-4be3-baa0-03e1fcbd5f79 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:02.814650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6911232b-89f0-4be3-baa0-03e1fcbd5f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:02.834270 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:02.891891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6900527-8941-4497-913c-767aa387f5a0 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:02.893287 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 533/1037] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:30:02.917047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911232b-89f0-4be3-baa0-03e1fcbd5f79 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:02.928332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559c2bc-8605-461a-aa6f-ed39e54551e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:02.928990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559c2bc-8605-461a-aa6f-ed39e54551e6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:02.929269 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559c2bc-8605-461a-aa6f-ed39e54551e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:02.929545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559c2bc-8605-461a-aa6f-ed39e54551e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:02.935802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:03.003771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1559c2bc-8605-461a-aa6f-ed39e54551e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:03.005358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1559c2bc-8605-461a-aa6f-ed39e54551e6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:03.020896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911232b-89f0-4be3-baa0-03e1fcbd5f79 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:03.055950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b15533-bff7-4ab3-8d9d-743eec11b73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:03.056798 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b15533-bff7-4ab3-8d9d-743eec11b73b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:03.057085 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b15533-bff7-4ab3-8d9d-743eec11b73b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:03.057357 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b15533-bff7-4ab3-8d9d-743eec11b73b None admin] PATH_INFO: `/v3/services/image` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:03.058028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b15533-bff7-4ab3-8d9d-743eec11b73b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:03.059400 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b15533-bff7-4ab3-8d9d-743eec11b73b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:03.060335 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:03.071466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-87b15533-bff7-4ab3-8d9d-743eec11b73b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:30:03.071466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:03.071466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:30:03.071466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:30:03.071466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:30:03.071466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:30:03.071466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:30:03.071466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:30:03.071466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:30:03.071466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:30:03.071466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:30:03.071466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:30:03.071466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:03.071466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:03.071466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:03.071466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:03.071466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:03.072995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:03.072995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:03.072995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:03.072995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:03.072995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:03.072995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:03.072995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:30:03.072995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:30:03.072995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:30:03.072995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:30:03.072995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:03.072995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:03.072995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:03.072995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:03.072995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:03.072995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:03.074075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:03.074075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:03.074075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:03.074075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:03.074075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:03.074075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:03.074075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:03.074075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:03.074075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:03.074075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:03.074075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:30:03.074075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:30:03.074075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:30:03.074075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:30:03.074075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 534/1038] 10.222.0.52 () {56 vars in 1105 bytes} [Tue Jul 12 13:30:02 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:30:03.079029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b15533-bff7-4ab3-8d9d-743eec11b73b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:03.113008 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 505/1039] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:30:02 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:30:03.118931 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559c2bc-8605-461a-aa6f-ed39e54551e6 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:03.162384 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:03.182404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b15533-bff7-4ab3-8d9d-743eec11b73b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:03.219241 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b220897-be2a-4e38-b937-64382d1b86c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:03.220025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b220897-be2a-4e38-b937-64382d1b86c2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:03.220243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b220897-be2a-4e38-b937-64382d1b86c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:03.220463 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b220897-be2a-4e38-b937-64382d1b86c2 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:03.221162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b220897-be2a-4e38-b937-64382d1b86c2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:03.221558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b220897-be2a-4e38-b937-64382d1b86c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:03.222861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b220897-be2a-4e38-b937-64382d1b86c2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:03.223682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559c2bc-8605-461a-aa6f-ed39e54551e6 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:03.232356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 535/1040] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:30:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:03.233379 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b220897-be2a-4e38-b937-64382d1b86c2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:03.263297 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b4972-d153-4742-81b0-4d7f221a658f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:03.264090 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b4972-d153-4742-81b0-4d7f221a658f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:03.264371 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b4972-d153-4742-81b0-4d7f221a658f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:03.264642 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b4972-d153-4742-81b0-4d7f221a658f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:03.275039 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b4972-d153-4742-81b0-4d7f221a658f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:03.277178 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b4972-d153-4742-81b0-4d7f221a658f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:03.295472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-c90b4972-d153-4742-81b0-4d7f221a658f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:30:03.295472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:03.295472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:03.295472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:03.295472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:03.295472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:03.295472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:03.295472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:03.295472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:03.295472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:03.295472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:03.295472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:03.295472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:30:03.295472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:30:03.295472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:30:03.296530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:30:03.296530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:03.296530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:03.296530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:03.296530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:03.296530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:03.296530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:03.296530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:03.296530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:03.296530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:03.296530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:03.296530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:03.296530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:03.296530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:03.296530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:03.296530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:03.296530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:03.297446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:30:03.297446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:30:03.297446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:30:03.297446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:03.297446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 506/1041] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:30:03.302248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b4972-d153-4742-81b0-4d7f221a658f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:03.311152 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:03.346442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b220897-be2a-4e38-b937-64382d1b86c2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:03.386771 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070eb0f2-67a6-44ff-b5cd-47b40f7e0433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:03.387438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eb0f2-67a6-44ff-b5cd-47b40f7e0433 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:03.387726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eb0f2-67a6-44ff-b5cd-47b40f7e0433 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:03.388007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eb0f2-67a6-44ff-b5cd-47b40f7e0433 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:03.388712 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070eb0f2-67a6-44ff-b5cd-47b40f7e0433 None admin] RBAC: Adding query filter params (type=image) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:03.389019 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070eb0f2-67a6-44ff-b5cd-47b40f7e0433 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:03.390439 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070eb0f2-67a6-44ff-b5cd-47b40f7e0433 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:03.395899 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 536/1042] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:30:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:03.402106 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070eb0f2-67a6-44ff-b5cd-47b40f7e0433 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:03.408380 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b4972-d153-4742-81b0-4d7f221a658f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:03.415299 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:03.449933 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9886dd4-1ab0-43c5-af7d-a4d77d3f48e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:03.450635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9886dd4-1ab0-43c5-af7d-a4d77d3f48e8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:03.450910 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9886dd4-1ab0-43c5-af7d-a4d77d3f48e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:03.451184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9886dd4-1ab0-43c5-af7d-a4d77d3f48e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:03.451901 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9886dd4-1ab0-43c5-af7d-a4d77d3f48e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:03.452205 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9886dd4-1ab0-43c5-af7d-a4d77d3f48e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:03.453575 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9886dd4-1ab0-43c5-af7d-a4d77d3f48e8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:03.466053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 507/1043] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:03.471478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9886dd4-1ab0-43c5-af7d-a4d77d3f48e8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:03.520375 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070eb0f2-67a6-44ff-b5cd-47b40f7e0433 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:03.545242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:03.562052 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ac61b-0ecb-43d1-a868-b8e82e6e4e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:03.562720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ac61b-0ecb-43d1-a868-b8e82e6e4e14 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:03.563001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ac61b-0ecb-43d1-a868-b8e82e6e4e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:03.563274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ac61b-0ecb-43d1-a868-b8e82e6e4e14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:03.564007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ac61b-0ecb-43d1-a868-b8e82e6e4e14 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:03.565376 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ac61b-0ecb-43d1-a868-b8e82e6e4e14 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:03.584822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9886dd4-1ab0-43c5-af7d-a4d77d3f48e8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:03.585335 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 537/1044] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jul 12 13:30:03 2022] POST /identity/v3/endpoints => generated 327 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:30:03.630339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3def76-5f82-4a04-8168-26f785a8d2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:03.631145 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3def76-5f82-4a04-8168-26f785a8d2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:03.631447 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3def76-5f82-4a04-8168-26f785a8d2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:03.631699 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3def76-5f82-4a04-8168-26f785a8d2e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:03.639973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3def76-5f82-4a04-8168-26f785a8d2e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:03.641639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3def76-5f82-4a04-8168-26f785a8d2e7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:03.657405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-9b3def76-5f82-4a04-8168-26f785a8d2e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:30:03.657405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:03.657405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:03.657405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:03.657405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:03.657405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:03.657405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:03.657405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:03.657405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:03.657405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:03.657405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:03.657405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:03.657405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:30:03.657405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:30:03.657405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:30:03.658443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:30:03.658443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:03.658443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:03.658443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:03.658443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:03.658443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:03.658443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:03.658443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:03.658443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:03.658443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:03.658443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:03.658443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:03.658443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:03.658443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:03.658443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:03.658443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:03.658443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:03.659353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:30:03.659353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:30:03.659353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:30:03.659353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:30:03.659353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:30:03.659353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:30:03.659353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:30:03.659353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:03.659353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 508/1045] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:30:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:30:03.664451 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ac61b-0ecb-43d1-a868-b8e82e6e4e14 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:03.735475 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:03.769386 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f89d7e-b7cd-49f1-9e61-46c69e768e79 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:03.769670 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f89d7e-b7cd-49f1-9e61-46c69e768e79 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:03.769946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f89d7e-b7cd-49f1-9e61-46c69e768e79 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:03.771379 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 509/1046] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:30:03.776665 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795496df-7fae-4176-8ef5-862b6c2e700b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:03.776913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795496df-7fae-4176-8ef5-862b6c2e700b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:03.777195 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795496df-7fae-4176-8ef5-862b6c2e700b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:03.777585 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ac61b-0ecb-43d1-a868-b8e82e6e4e14 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:03.794439 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-795496df-7fae-4176-8ef5-862b6c2e700b None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:03.795653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-795496df-7fae-4176-8ef5-862b6c2e700b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:03.816415 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf78cf4-f024-438c-af8f-6eeb23c800ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:03.817087 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf78cf4-f024-438c-af8f-6eeb23c800ec None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:03.817375 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf78cf4-f024-438c-af8f-6eeb23c800ec None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:03.817976 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf78cf4-f024-438c-af8f-6eeb23c800ec None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:03.818468 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf78cf4-f024-438c-af8f-6eeb23c800ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:03.818786 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf78cf4-f024-438c-af8f-6eeb23c800ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:03.820248 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf78cf4-f024-438c-af8f-6eeb23c800ec None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:03.832816 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 538/1047] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:30:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:03.838844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf78cf4-f024-438c-af8f-6eeb23c800ec None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:03.879037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 510/1048] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:30:03.903361 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795496df-7fae-4176-8ef5-862b6c2e700b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:03.948473 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf78cf4-f024-438c-af8f-6eeb23c800ec None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:03.962925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:03.980205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:03.990798 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72838b-c828-4621-81a8-c2a18a18cae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:03.991479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72838b-c828-4621-81a8-c2a18a18cae7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:03.991765 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72838b-c828-4621-81a8-c2a18a18cae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:03.992179 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72838b-c828-4621-81a8-c2a18a18cae7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:03.993132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72838b-c828-4621-81a8-c2a18a18cae7 None admin] RBAC: Adding query filter params (role.id=19deffe64f224b68880ce003b4af8b2b, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=0d3915cde0d04bcfb9b5ce8c89af1e5b) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:03.993447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72838b-c828-4621-81a8-c2a18a18cae7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:03.995468 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72838b-c828-4621-81a8-c2a18a18cae7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:04.009998 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 539/1049] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:30:03 2022] GET /identity/v3/role_assignments?user.id=0d3915cde0d04bcfb9b5ce8c89af1e5b&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=19deffe64f224b68880ce003b4af8b2b => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:04.022709 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795496df-7fae-4176-8ef5-862b6c2e700b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:04.063746 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913305d-18fb-4194-8a2e-efdae8e7d6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:04.064561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913305d-18fb-4194-8a2e-efdae8e7d6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:04.064845 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913305d-18fb-4194-8a2e-efdae8e7d6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:04.065116 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913305d-18fb-4194-8a2e-efdae8e7d6e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:04.066382 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913305d-18fb-4194-8a2e-efdae8e7d6e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:04.067844 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913305d-18fb-4194-8a2e-efdae8e7d6e5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:04.069724 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 511/1050] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:30:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:04.075530 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72838b-c828-4621-81a8-c2a18a18cae7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:04.187955 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72838b-c828-4621-81a8-c2a18a18cae7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:04.222059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e4837-03f3-4408-b27b-47dab131ebfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:04.222758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e4837-03f3-4408-b27b-47dab131ebfa None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:04.223042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e4837-03f3-4408-b27b-47dab131ebfa None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:04.223316 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e4837-03f3-4408-b27b-47dab131ebfa None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:04.224046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e4837-03f3-4408-b27b-47dab131ebfa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:04.225490 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e4837-03f3-4408-b27b-47dab131ebfa None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:04.254080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 540/1051] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:30:04 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:30:05.658022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f1255-76c0-45c0-84f9-03ce6b6fc8e9 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:05.658615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f1255-76c0-45c0-84f9-03ce6b6fc8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:05.658615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f1255-76c0-45c0-84f9-03ce6b6fc8e9 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:05.660079 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 512/1052] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:05.665052 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2517fd-e319-4a1f-a82a-6283857be2aa None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:05.665466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2517fd-e319-4a1f-a82a-6283857be2aa None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:05.665632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2517fd-e319-4a1f-a82a-6283857be2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:05.684080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-df2517fd-e319-4a1f-a82a-6283857be2aa None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:05.685271 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-df2517fd-e319-4a1f-a82a-6283857be2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:05.728132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:05.758108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 541/1053] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:05.777884 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f1255-76c0-45c0-84f9-03ce6b6fc8e9 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:05.889108 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f1255-76c0-45c0-84f9-03ce6b6fc8e9 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:05.909203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:05.923751 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fddeda-8edf-4b4b-84bb-dcc0c4458b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:05.924677 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fddeda-8edf-4b4b-84bb-dcc0c4458b4c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:05.925135 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fddeda-8edf-4b4b-84bb-dcc0c4458b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:05.925599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fddeda-8edf-4b4b-84bb-dcc0c4458b4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:05.992351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fddeda-8edf-4b4b-84bb-dcc0c4458b4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:05.994716 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fddeda-8edf-4b4b-84bb-dcc0c4458b4c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:06.035499 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d88c69-1fc0-487a-8823-7af7a4d94de4 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:06.035787 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d88c69-1fc0-487a-8823-7af7a4d94de4 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:06.036277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d88c69-1fc0-487a-8823-7af7a4d94de4 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:06.037645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 542/1054] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:06.043285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ab1e3-17e3-4812-b651-0ec00d584005 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:06.043575 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ab1e3-17e3-4812-b651-0ec00d584005 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:06.043847 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ab1e3-17e3-4812-b651-0ec00d584005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:06.062720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-442ab1e3-17e3-4812-b651-0ec00d584005 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:06.064130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-442ab1e3-17e3-4812-b651-0ec00d584005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:06.101818 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 513/1055] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:30:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:30:06.108131 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fddeda-8edf-4b4b-84bb-dcc0c4458b4c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:06.134165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:06.146125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 543/1056] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:30:06.169389 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ab1e3-17e3-4812-b651-0ec00d584005 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:06.221126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fddeda-8edf-4b4b-84bb-dcc0c4458b4c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:06.259565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fac4a6-b982-4904-bdf8-3dd5e7ed8c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:06.260357 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fac4a6-b982-4904-bdf8-3dd5e7ed8c18 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:06.260641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fac4a6-b982-4904-bdf8-3dd5e7ed8c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:06.260916 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fac4a6-b982-4904-bdf8-3dd5e7ed8c18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:06.271654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fac4a6-b982-4904-bdf8-3dd5e7ed8c18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:06.273115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fac4a6-b982-4904-bdf8-3dd5e7ed8c18 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:06.277037 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ab1e3-17e3-4812-b651-0ec00d584005 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:06.285806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:06.291767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-41fac4a6-b982-4904-bdf8-3dd5e7ed8c18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:30:06.291767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:06.291767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:06.291767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:06.291767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:06.291767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:06.291767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:06.291767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:06.291767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:06.291767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:06.291767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:06.291767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:06.291767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:30:06.291767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:30:06.291767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:30:06.292667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:30:06.292667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:06.292667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:06.292667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:06.292667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:06.292667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:06.292667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:06.292667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:06.292667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:06.292667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:06.292667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:06.292667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:06.292667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:06.292667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:06.292667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:06.292667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:06.292667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:06.293566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:30:06.293566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:30:06.293566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:30:06.293566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:06.293566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 514/1057] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:06.298799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fac4a6-b982-4904-bdf8-3dd5e7ed8c18 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:06.320067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58de64-de0d-4605-aa29-8bba1b018f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:06.320900 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58de64-de0d-4605-aa29-8bba1b018f54 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:06.321186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58de64-de0d-4605-aa29-8bba1b018f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:06.321465 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58de64-de0d-4605-aa29-8bba1b018f54 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:06.333708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58de64-de0d-4605-aa29-8bba1b018f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:30:06.342149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-7f58de64-de0d-4605-aa29-8bba1b018f54 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:30:06.342149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:06.342149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:06.342149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:06.342149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:06.342149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:06.342149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:06.342149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:06.342149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:06.342149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:06.342149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:06.342149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:06.342149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:30:06.342149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:30:06.342149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:30:06.342149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:30:06.343189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:30:06.343189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:30:06.343189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:30:06.343189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:30:06.343189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:06.343189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:06.343189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:06.343189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:06.343189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:06.343189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:06.343189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:06.343189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:06.343189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:06.343189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:06.343189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:06.343189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:06.344084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:06.344084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:06.344084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:06.344084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:06.344084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:30:06.344084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:30:06.344084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:30:06.344084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:30:06.344084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:30:06.344084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:30:06.344084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:30:06.344084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:30:06.346104 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 544/1058] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Jul 12 13:30:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:30:06.347586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbc7e4-fc56-4e89-90a6-74fcbbeaef7f None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:06.347882 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbc7e4-fc56-4e89-90a6-74fcbbeaef7f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:06.348320 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbc7e4-fc56-4e89-90a6-74fcbbeaef7f None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:06.349566 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 545/1059] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:06.350165 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bbc7e4-fc56-4e89-90a6-74fcbbeaef7f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:06.376302 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:06.406723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fac4a6-b982-4904-bdf8-3dd5e7ed8c18 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:06.441391 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3d5f7-589b-4f49-9093-91289ffeeb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:06.441992 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3d5f7-589b-4f49-9093-91289ffeeb0b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:06.442276 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3d5f7-589b-4f49-9093-91289ffeeb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:06.442551 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3d5f7-589b-4f49-9093-91289ffeeb0b None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:06.443259 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3d5f7-589b-4f49-9093-91289ffeeb0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:06.443563 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3d5f7-589b-4f49-9093-91289ffeeb0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:06.444925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3d5f7-589b-4f49-9093-91289ffeeb0b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:06.455003 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bbc7e4-fc56-4e89-90a6-74fcbbeaef7f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:06.456458 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 515/1060] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:06.457668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e9971-8ee7-4ddb-b50f-d67991632e5d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:06.457952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e9971-8ee7-4ddb-b50f-d67991632e5d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:06.458252 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e9971-8ee7-4ddb-b50f-d67991632e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:06.477133 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a80e9971-8ee7-4ddb-b50f-d67991632e5d None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:06.479170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a80e9971-8ee7-4ddb-b50f-d67991632e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:06.502476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d6bc6-a829-4795-af43-23595d271702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:06.503291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d6bc6-a829-4795-af43-23595d271702 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:06.503579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d6bc6-a829-4795-af43-23595d271702 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:06.503917 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d6bc6-a829-4795-af43-23595d271702 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:06.505190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d6bc6-a829-4795-af43-23595d271702 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:06.505326 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d6bc6-a829-4795-af43-23595d271702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:06.506970 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d6bc6-a829-4795-af43-23595d271702 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:06.522366 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 546/1061] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Jul 12 13:30:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:06.523590 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d6bc6-a829-4795-af43-23595d271702 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:06.559410 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:06.566675 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 516/1062] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:06.597604 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e9971-8ee7-4ddb-b50f-d67991632e5d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:06.658241 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d6bc6-a829-4795-af43-23595d271702 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:06.672603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:06.708112 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b8eb3-4edd-4ad4-ac50-f9661aee7f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:06.709203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b8eb3-4edd-4ad4-ac50-f9661aee7f1b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:06.709203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b8eb3-4edd-4ad4-ac50-f9661aee7f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:06.709368 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b8eb3-4edd-4ad4-ac50-f9661aee7f1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:06.723400 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e9971-8ee7-4ddb-b50f-d67991632e5d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:06.723712 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b8eb3-4edd-4ad4-ac50-f9661aee7f1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:06.725202 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b8eb3-4edd-4ad4-ac50-f9661aee7f1b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:06.743949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-128b8eb3-4edd-4ad4-ac50-f9661aee7f1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:30:06.743949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:06.743949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:06.743949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:06.743949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:06.743949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:06.743949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:06.743949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:06.743949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:06.743949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:06.743949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:06.743949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:06.743949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:30:06.743949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:30:06.743949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:30:06.744936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:30:06.744936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:06.744936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:06.744936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:06.744936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:06.744936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:06.744936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:06.744936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:06.744936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:06.744936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:06.744936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:06.744936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:06.744936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:06.744936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:06.744936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:06.744936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:06.744936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:06.745848 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:30:06.745848 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:30:06.745848 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:30:06.745848 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:30:06.745848 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 547/1063] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:06.751232 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b8eb3-4edd-4ad4-ac50-f9661aee7f1b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:06.764379 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8d52f-0c65-4975-b47e-d1481bbefff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:06.765174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8d52f-0c65-4975-b47e-d1481bbefff4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:06.765455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8d52f-0c65-4975-b47e-d1481bbefff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:06.765725 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8d52f-0c65-4975-b47e-d1481bbefff4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:06.776649 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8d52f-0c65-4975-b47e-d1481bbefff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:30:06.784415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-77a8d52f-0c65-4975-b47e-d1481bbefff4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:06.784415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:06.784415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:06.784415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:06.784415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:06.784415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:06.784415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:06.784415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:06.784415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:06.784415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:06.784415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:06.784415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:06.784415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:30:06.784415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:30:06.784415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:30:06.784415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:30:06.785589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:30:06.785589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:30:06.785589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:30:06.785589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:30:06.785589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:06.785589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:06.785589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:06.785589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:06.785589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:06.785589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:06.785589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:06.785589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:06.785589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:06.785589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:06.785589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:06.785589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:06.786639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:06.786639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:06.786639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:06.786639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:06.786639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:30:06.786639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:30:06.786639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:30:06.786639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:30:06.786639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:30:06.786639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:30:06.786639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:06.786639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:06.786639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 517/1064] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:30:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:30:06.791443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8d52f-0c65-4975-b47e-d1481bbefff4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:06.835621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:06.856862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b8eb3-4edd-4ad4-ac50-f9661aee7f1b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:06.890828 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd79c5-e4e9-4fc1-af8d-202b998f6dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:06.891527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd79c5-e4e9-4fc1-af8d-202b998f6dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:06.891813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd79c5-e4e9-4fc1-af8d-202b998f6dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:06.892089 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd79c5-e4e9-4fc1-af8d-202b998f6dc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:06.892816 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd79c5-e4e9-4fc1-af8d-202b998f6dc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:06.893125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd79c5-e4e9-4fc1-af8d-202b998f6dc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:06.894544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd79c5-e4e9-4fc1-af8d-202b998f6dc9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:06.894825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8d52f-0c65-4975-b47e-d1481bbefff4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:06.907758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 548/1065] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:06.914042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd79c5-e4e9-4fc1-af8d-202b998f6dc9 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:06.919052 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:06.931534 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75e4cb-1309-41ab-96b7-438e71d6a9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:06.932206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75e4cb-1309-41ab-96b7-438e71d6a9af None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:06.932494 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75e4cb-1309-41ab-96b7-438e71d6a9af None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:06.932765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75e4cb-1309-41ab-96b7-438e71d6a9af None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:06.933498 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75e4cb-1309-41ab-96b7-438e71d6a9af None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:06.933801 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75e4cb-1309-41ab-96b7-438e71d6a9af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:06.935238 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75e4cb-1309-41ab-96b7-438e71d6a9af None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:06.944750 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 518/1066] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:30:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:06.951205 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75e4cb-1309-41ab-96b7-438e71d6a9af None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:07.027923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd79c5-e4e9-4fc1-af8d-202b998f6dc9 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:07.055408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75e4cb-1309-41ab-96b7-438e71d6a9af None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:07.061150 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e3a87-bd6d-4c3e-8972-6d82c66d7a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:07.061950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e3a87-bd6d-4c3e-8972-6d82c66d7a98 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:07.062271 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e3a87-bd6d-4c3e-8972-6d82c66d7a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:07.062550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e3a87-bd6d-4c3e-8972-6d82c66d7a98 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:07.069590 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e3a87-bd6d-4c3e-8972-6d82c66d7a98 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:07.071072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e3a87-bd6d-4c3e-8972-6d82c66d7a98 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:07.074259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:07.086550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-eb4e3a87-bd6d-4c3e-8972-6d82c66d7a98 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:30:07.086550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:07.086550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:07.086550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:07.086550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:07.086550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:07.086550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:07.086550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:07.086550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:07.086550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:07.086550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:07.086550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:07.086550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:30:07.086550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:30:07.086550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:30:07.086550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:07.087563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:07.089049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:07.089049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:07.089049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:30:07.089049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:30:07.089049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:30:07.089049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:30:07.089049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:30:07.089049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:30:07.089049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:30:07.089049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:30:07.089049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 549/1067] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jul 12 13:30:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:30:07.090555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239e1ca-ee3b-49e1-bea2-7452a8a65bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:07.091432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239e1ca-ee3b-49e1-bea2-7452a8a65bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:07.091714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239e1ca-ee3b-49e1-bea2-7452a8a65bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:07.091985 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239e1ca-ee3b-49e1-bea2-7452a8a65bf0 None admin] PATH_INFO: `/v3/users/50a99119d9374da79b917ceb19264ccf` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:07.094364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e3a87-bd6d-4c3e-8972-6d82c66d7a98 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:07.105907 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239e1ca-ee3b-49e1-bea2-7452a8a65bf0 None admin] RBAC: Authorizing `identity:get_user(user_id=50a99119d9374da79b917ceb19264ccf)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:07.107512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239e1ca-ee3b-49e1-bea2-7452a8a65bf0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:07.109204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 519/1068] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:30:06 2022] GET /identity/v3/users/50a99119d9374da79b917ceb19264ccf => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:07.114952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239e1ca-ee3b-49e1-bea2-7452a8a65bf0 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:07.157636 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:07.192978 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e3a87-bd6d-4c3e-8972-6d82c66d7a98 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:07.214054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239e1ca-ee3b-49e1-bea2-7452a8a65bf0 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:07.226603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f2b16-c6e9-4a11-99fa-cce4a607d5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:07.227276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f2b16-c6e9-4a11-99fa-cce4a607d5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:07.227557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f2b16-c6e9-4a11-99fa-cce4a607d5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:07.227838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f2b16-c6e9-4a11-99fa-cce4a607d5c4 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:07.228670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f2b16-c6e9-4a11-99fa-cce4a607d5c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:07.228980 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f2b16-c6e9-4a11-99fa-cce4a607d5c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:07.230457 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f2b16-c6e9-4a11-99fa-cce4a607d5c4 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:07.247512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488308b-fc2a-4984-8dde-90ccfeaab7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:07.248155 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488308b-fc2a-4984-8dde-90ccfeaab7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:07.248433 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488308b-fc2a-4984-8dde-90ccfeaab7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:07.248705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488308b-fc2a-4984-8dde-90ccfeaab7e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:07.250119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 550/1069] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jul 12 13:30:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:07.253670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f2b16-c6e9-4a11-99fa-cce4a607d5c4 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:07.313989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:07.327514 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488308b-fc2a-4984-8dde-90ccfeaab7e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:07.328970 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488308b-fc2a-4984-8dde-90ccfeaab7e8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:07.360825 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f2b16-c6e9-4a11-99fa-cce4a607d5c4 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:07.380236 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:07.400674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50666ea5-c976-4d34-a32b-8babc07dc78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:07.401467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50666ea5-c976-4d34-a32b-8babc07dc78f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:07.401751 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50666ea5-c976-4d34-a32b-8babc07dc78f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:07.402053 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50666ea5-c976-4d34-a32b-8babc07dc78f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:07.412068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50666ea5-c976-4d34-a32b-8babc07dc78f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:07.413524 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50666ea5-c976-4d34-a32b-8babc07dc78f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:07.431051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-50666ea5-c976-4d34-a32b-8babc07dc78f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:30:07.431051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:07.431051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:07.431051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:07.431051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:07.431051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:07.431051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:07.431051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:07.431051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:07.431051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:07.431051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:07.431051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:07.431051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:30:07.431051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:30:07.431051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:30:07.432512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:30:07.432512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:07.432512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:07.432512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:07.432512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:07.432512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:07.432512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:07.432512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:07.432512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:07.432512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:07.432512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:07.432512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:07.432512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:07.432512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:07.432512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:07.432512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:07.432512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:07.433523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:30:07.433523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:30:07.433523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:30:07.433523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:30:07.433523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 551/1070] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:30:07.437938 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 520/1071] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:30:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 13:30:07.438358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50666ea5-c976-4d34-a32b-8babc07dc78f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:07.443303 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488308b-fc2a-4984-8dde-90ccfeaab7e8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:07.537524 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50666ea5-c976-4d34-a32b-8babc07dc78f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:07.541571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488308b-fc2a-4984-8dde-90ccfeaab7e8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:07.548746 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:07.574254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5815a-12ad-4582-b805-f5d04791907b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:07.574834 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5815a-12ad-4582-b805-f5d04791907b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:07.575118 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5815a-12ad-4582-b805-f5d04791907b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:07.575395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5815a-12ad-4582-b805-f5d04791907b None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:07.576115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5815a-12ad-4582-b805-f5d04791907b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:07.576425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5815a-12ad-4582-b805-f5d04791907b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:07.577800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5815a-12ad-4582-b805-f5d04791907b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:07.578585 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a72a1-6f11-4529-a0c8-ad7d74ba9e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:07.579446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a72a1-6f11-4529-a0c8-ad7d74ba9e7e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:07.579727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a72a1-6f11-4529-a0c8-ad7d74ba9e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:07.580004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a72a1-6f11-4529-a0c8-ad7d74ba9e7e None admin] PATH_INFO: `/v3/projects/97655558670b4282aec6d5144211f188` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:07.581181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a72a1-6f11-4529-a0c8-ad7d74ba9e7e None admin] RBAC: Authorizing `identity:get_project(project_id=97655558670b4282aec6d5144211f188)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:07.582626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a72a1-6f11-4529-a0c8-ad7d74ba9e7e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:07.584329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 521/1072] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:30:07 2022] GET /identity/v3/projects/97655558670b4282aec6d5144211f188 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:07.589169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 552/1073] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:07.590472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a72a1-6f11-4529-a0c8-ad7d74ba9e7e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:07.594403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5815a-12ad-4582-b805-f5d04791907b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:07.602222 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:07.692395 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a72a1-6f11-4529-a0c8-ad7d74ba9e7e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:07.695326 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5815a-12ad-4582-b805-f5d04791907b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:07.729267 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3707d2a-35ef-439d-b30c-7af9353f7632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:07.729774 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ad935-7e39-44e1-886e-8d6e97e441b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:07.730257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3707d2a-35ef-439d-b30c-7af9353f7632 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:07.730257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3707d2a-35ef-439d-b30c-7af9353f7632 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:07.730483 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3707d2a-35ef-439d-b30c-7af9353f7632 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:07.730641 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ad935-7e39-44e1-886e-8d6e97e441b7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:07.730955 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ad935-7e39-44e1-886e-8d6e97e441b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:07.731138 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ad935-7e39-44e1-886e-8d6e97e441b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:07.731299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3707d2a-35ef-439d-b30c-7af9353f7632 None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=97655558670b4282aec6d5144211f188, user.id=50a99119d9374da79b917ceb19264ccf) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:07.731607 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3707d2a-35ef-439d-b30c-7af9353f7632 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:07.733028 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3707d2a-35ef-439d-b30c-7af9353f7632 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:07.737910 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ad935-7e39-44e1-886e-8d6e97e441b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:07.738436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 522/1074] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:30:07 2022] GET /identity/v3/role_assignments?user.id=50a99119d9374da79b917ceb19264ccf&scope.project.id=97655558670b4282aec6d5144211f188&role.id=864a6966630544e8bd68bcafba905c6f => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:07.739421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ad935-7e39-44e1-886e-8d6e97e441b7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:07.755401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-cd1ad935-7e39-44e1-886e-8d6e97e441b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:30:07.755401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:07.755401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:07.755401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:07.755401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:07.755401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:07.755401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:07.755401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:07.755401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:07.755401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:07.755401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:07.755401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:07.755401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:30:07.755401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:30:07.755401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:30:07.756568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:30:07.756568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:07.756568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:07.756568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:07.756568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:07.756568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:07.756568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:07.756568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:07.756568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:07.756568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:07.756568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:07.756568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:07.756568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:07.756568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:07.756568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:07.756568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:07.756568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:07.757674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:30:07.757674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:30:07.757674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:30:07.757674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:30:07.757674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:30:07.757674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:30:07.757674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:30:07.757674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:30:07.757674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 553/1075] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:30:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:30:07.762827 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3707d2a-35ef-439d-b30c-7af9353f7632 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:07.841442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:07.875301 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3707d2a-35ef-439d-b30c-7af9353f7632 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:07.910756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3bff7-efed-4024-91fe-fbecc71b159c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:07.911416 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3bff7-efed-4024-91fe-fbecc71b159c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:07.911698 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3bff7-efed-4024-91fe-fbecc71b159c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:07.911971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3bff7-efed-4024-91fe-fbecc71b159c None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:07.912749 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3bff7-efed-4024-91fe-fbecc71b159c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:07.913056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3bff7-efed-4024-91fe-fbecc71b159c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:07.914519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3bff7-efed-4024-91fe-fbecc71b159c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:07.926644 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 523/1076] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:30:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:07.932666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ad935-7e39-44e1-886e-8d6e97e441b7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:07.962169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:08.037443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ad935-7e39-44e1-886e-8d6e97e441b7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:08.072628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c992c28-40bb-4a38-9689-291ce1b2e967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:08.073525 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c992c28-40bb-4a38-9689-291ce1b2e967 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:08.073803 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c992c28-40bb-4a38-9689-291ce1b2e967 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:08.074120 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c992c28-40bb-4a38-9689-291ce1b2e967 None admin] PATH_INFO: `/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/0d3915cde0d04bcfb9b5ce8c89af1e5b/roles/19deffe64f224b68880ce003b4af8b2b` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:08.076316 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c992c28-40bb-4a38-9689-291ce1b2e967 None admin] RBAC: Authorizing `identity:create_grant(project_id=36aa0326fd0545dface07a9ddc308ccd, user_id=0d3915cde0d04bcfb9b5ce8c89af1e5b, role_id=19deffe64f224b68880ce003b4af8b2b)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:08.077764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c992c28-40bb-4a38-9689-291ce1b2e967 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:08.091068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 554/1077] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:30:07 2022] PUT /identity/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/0d3915cde0d04bcfb9b5ce8c89af1e5b/roles/19deffe64f224b68880ce003b4af8b2b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:30:09.734955 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0193e9-ca22-4d85-a913-d4a74d8c0975 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:09.735513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0193e9-ca22-4d85-a913-d4a74d8c0975 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:09.735513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0193e9-ca22-4d85-a913-d4a74d8c0975 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:09.736887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 524/1078] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:09.742485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec87cd-ba98-4511-81e3-4efc27483589 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:09.742959 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec87cd-ba98-4511-81e3-4efc27483589 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:09.743312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec87cd-ba98-4511-81e3-4efc27483589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:09.761184 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-89ec87cd-ba98-4511-81e3-4efc27483589 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:09.762393 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-89ec87cd-ba98-4511-81e3-4efc27483589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:09.827467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 555/1079] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:09.849165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0193e9-ca22-4d85-a913-d4a74d8c0975 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:09.950931 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0193e9-ca22-4d85-a913-d4a74d8c0975 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:09.983861 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39562885-ad28-47f5-b8f4-570c279c7216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:09.984644 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39562885-ad28-47f5-b8f4-570c279c7216 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:09.984925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39562885-ad28-47f5-b8f4-570c279c7216 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:09.985247 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39562885-ad28-47f5-b8f4-570c279c7216 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:09.987052 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:09.996875 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39562885-ad28-47f5-b8f4-570c279c7216 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:30:10.004822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-39562885-ad28-47f5-b8f4-570c279c7216 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:10.004822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:10.004822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:10.004822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:10.004822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:10.004822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:10.004822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:10.004822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:10.004822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:10.004822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:10.004822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:10.004822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:10.004822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:30:10.004822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:30:10.004822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:30:10.004822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:30:10.005752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:30:10.005752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:30:10.005752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:30:10.005752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:30:10.005752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:10.005752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:10.005752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:10.005752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:10.005752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:10.005752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:10.005752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:10.005752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:10.005752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:10.005752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:10.005752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:10.005752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:10.006640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:10.006640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:10.006640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:10.006640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:10.006640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:30:10.006640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:30:10.006640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:30:10.006640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:30:10.006640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:30:10.006640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:30:10.006640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:10.006640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:10.006640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 525/1080] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:30:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:10.012524 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec87cd-ba98-4511-81e3-4efc27483589 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:10.032868 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:10.105512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce797b7-f615-42e7-b5a8-ad1a919c58d3 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:10.105794 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce797b7-f615-42e7-b5a8-ad1a919c58d3 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:10.106114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce797b7-f615-42e7-b5a8-ad1a919c58d3 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:10.107631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 526/1081] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:30:10.112211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d1922-f758-4ae1-9c1b-4cc29e249de4 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:10.112500 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d1922-f758-4ae1-9c1b-4cc29e249de4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:10.112775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d1922-f758-4ae1-9c1b-4cc29e249de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:10.114822 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec87cd-ba98-4511-81e3-4efc27483589 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:10.131302 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a01d1922-f758-4ae1-9c1b-4cc29e249de4 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:10.132491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a01d1922-f758-4ae1-9c1b-4cc29e249de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:10.150994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265f65c-7ff0-4f0c-9441-3570f25ab537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:10.151659 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265f65c-7ff0-4f0c-9441-3570f25ab537 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:10.151936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265f65c-7ff0-4f0c-9441-3570f25ab537 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:10.152208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265f65c-7ff0-4f0c-9441-3570f25ab537 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:10.152939 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265f65c-7ff0-4f0c-9441-3570f25ab537 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:10.153252 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265f65c-7ff0-4f0c-9441-3570f25ab537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:10.154648 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265f65c-7ff0-4f0c-9441-3570f25ab537 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:10.162511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 556/1082] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:30:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:10.169026 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265f65c-7ff0-4f0c-9441-3570f25ab537 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:10.199924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 527/1083] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:30:10.223355 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d1922-f758-4ae1-9c1b-4cc29e249de4 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:10.250110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:10.268160 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265f65c-7ff0-4f0c-9441-3570f25ab537 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:10.301615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afe1df-254f-49d0-a22c-ded055beb28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:10.302515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afe1df-254f-49d0-a22c-ded055beb28c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:10.302801 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afe1df-254f-49d0-a22c-ded055beb28c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:10.303076 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afe1df-254f-49d0-a22c-ded055beb28c None admin] PATH_INFO: `/v3/users/50a99119d9374da79b917ceb19264ccf` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:10.304275 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afe1df-254f-49d0-a22c-ded055beb28c None admin] RBAC: Authorizing `identity:get_user(user_id=50a99119d9374da79b917ceb19264ccf)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:10.305729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afe1df-254f-49d0-a22c-ded055beb28c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:10.307715 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 557/1084] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:30:10 2022] GET /identity/v3/users/50a99119d9374da79b917ceb19264ccf => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:10.313012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afe1df-254f-49d0-a22c-ded055beb28c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:10.324042 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d1922-f758-4ae1-9c1b-4cc29e249de4 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:10.348069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:10.360246 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8789a62-511d-41ec-bcf4-6389ea2d1ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:10.361057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8789a62-511d-41ec-bcf4-6389ea2d1ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:10.361339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8789a62-511d-41ec-bcf4-6389ea2d1ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:10.361617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8789a62-511d-41ec-bcf4-6389ea2d1ca8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:10.372087 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8789a62-511d-41ec-bcf4-6389ea2d1ca8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:30:10.380634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-b8789a62-511d-41ec-bcf4-6389ea2d1ca8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:30:10.380634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:10.380634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:10.380634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:10.380634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:10.380634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:10.380634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:10.380634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:10.380634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:10.380634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:10.380634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:10.380634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:10.380634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:30:10.380634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:30:10.380634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:30:10.380634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:30:10.381612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:30:10.381612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:30:10.381612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:30:10.381612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:30:10.381612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:10.381612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:10.381612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:10.381612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:10.381612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:10.381612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:10.381612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:10.381612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:10.381612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:10.381612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:10.381612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:10.381612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:10.382545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:10.382545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:10.382545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:10.382545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:10.382545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:30:10.382545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:30:10.382545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:30:10.382545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:30:10.382545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:30:10.382545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:30:10.382545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:30:10.382545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:10.382545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 528/1085] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Jul 12 13:30:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:30:10.387708 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8789a62-511d-41ec-bcf4-6389ea2d1ca8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:10.412511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afe1df-254f-49d0-a22c-ded055beb28c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:10.446213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59864eb-756b-461b-9dd6-a397ca38a710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:10.447068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59864eb-756b-461b-9dd6-a397ca38a710 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:10.447352 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59864eb-756b-461b-9dd6-a397ca38a710 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:10.447626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59864eb-756b-461b-9dd6-a397ca38a710 None admin] PATH_INFO: `/v3/projects/97655558670b4282aec6d5144211f188` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:10.448795 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59864eb-756b-461b-9dd6-a397ca38a710 None admin] RBAC: Authorizing `identity:get_project(project_id=97655558670b4282aec6d5144211f188)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:10.450266 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59864eb-756b-461b-9dd6-a397ca38a710 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:10.451945 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 558/1086] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:30:10 2022] GET /identity/v3/projects/97655558670b4282aec6d5144211f188 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:10.457704 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59864eb-756b-461b-9dd6-a397ca38a710 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:10.476540 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:10.486683 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8789a62-511d-41ec-bcf4-6389ea2d1ca8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:10.520351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75351d-2969-47af-8b88-e3c07cf496c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:10.520933 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75351d-2969-47af-8b88-e3c07cf496c3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:10.521215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75351d-2969-47af-8b88-e3c07cf496c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:10.521484 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75351d-2969-47af-8b88-e3c07cf496c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:10.522246 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75351d-2969-47af-8b88-e3c07cf496c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:10.522545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75351d-2969-47af-8b88-e3c07cf496c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:10.523954 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75351d-2969-47af-8b88-e3c07cf496c3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:10.532440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 529/1087] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Jul 12 13:30:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:10.538039 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75351d-2969-47af-8b88-e3c07cf496c3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:10.559996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59864eb-756b-461b-9dd6-a397ca38a710 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:10.582589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:10.594285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc4cf4-a0e1-4065-9503-35655c0bd1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:10.595176 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc4cf4-a0e1-4065-9503-35655c0bd1b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:10.595455 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc4cf4-a0e1-4065-9503-35655c0bd1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:10.595733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc4cf4-a0e1-4065-9503-35655c0bd1b1 None admin] PATH_INFO: `/v3/projects/97655558670b4282aec6d5144211f188/users/50a99119d9374da79b917ceb19264ccf/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:10.597833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc4cf4-a0e1-4065-9503-35655c0bd1b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=97655558670b4282aec6d5144211f188, user_id=50a99119d9374da79b917ceb19264ccf, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:10.599425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc4cf4-a0e1-4065-9503-35655c0bd1b1 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:10.613685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 559/1088] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:30:10 2022] PUT /identity/v3/projects/97655558670b4282aec6d5144211f188/users/50a99119d9374da79b917ceb19264ccf/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:30:10.639797 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75351d-2969-47af-8b88-e3c07cf496c3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:10.676772 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e4f66-f9ac-4d66-a45b-870fd52ed4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:10.677584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e4f66-f9ac-4d66-a45b-870fd52ed4bd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:10.677869 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e4f66-f9ac-4d66-a45b-870fd52ed4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:10.678175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e4f66-f9ac-4d66-a45b-870fd52ed4bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:10.691004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e4f66-f9ac-4d66-a45b-870fd52ed4bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:10.692504 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e4f66-f9ac-4d66-a45b-870fd52ed4bd None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:10.712061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-ff4e4f66-f9ac-4d66-a45b-870fd52ed4bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:30:10.712061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:10.712061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:10.712061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:10.712061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:10.712061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:10.712061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:10.712061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:10.712061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:10.712061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:10.712061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:10.712061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:10.712061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:30:10.712061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:30:10.712061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:30:10.713705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:30:10.713705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:10.713705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:10.713705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:10.713705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:10.713705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:10.713705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:10.713705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:10.713705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:10.713705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:10.713705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:10.713705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:10.713705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:10.713705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:10.713705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:10.713705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:10.713705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:10.714702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:30:10.714702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:30:10.714702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:30:10.714702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:10.714702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 530/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:30:10.720042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc4cf4-a0e1-4065-9503-35655c0bd1b1 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:10.804261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:10.827255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc4cf4-a0e1-4065-9503-35655c0bd1b1 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:10.862818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6c818-0446-4623-80d8-199f40bd28ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:10.863495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6c818-0446-4623-80d8-199f40bd28ef None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:10.863778 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6c818-0446-4623-80d8-199f40bd28ef None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:10.864058 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6c818-0446-4623-80d8-199f40bd28ef None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:10.864888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6c818-0446-4623-80d8-199f40bd28ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:10.865099 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6c818-0446-4623-80d8-199f40bd28ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:10.866520 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6c818-0446-4623-80d8-199f40bd28ef None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:10.878756 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 560/1090] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:10.884031 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e4f66-f9ac-4d66-a45b-870fd52ed4bd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:10.980192 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:10.984298 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e4f66-f9ac-4d66-a45b-870fd52ed4bd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:11.019544 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6218ba4-8f15-48f9-afb3-ee13235440eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:11.020342 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6218ba4-8f15-48f9-afb3-ee13235440eb None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:11.020626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6218ba4-8f15-48f9-afb3-ee13235440eb None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:11.020898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6218ba4-8f15-48f9-afb3-ee13235440eb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:11.028517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6218ba4-8f15-48f9-afb3-ee13235440eb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:11.029968 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6218ba4-8f15-48f9-afb3-ee13235440eb None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:11.043736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-a6218ba4-8f15-48f9-afb3-ee13235440eb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:30:11.043736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:11.043736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:11.043736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:11.043736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:11.043736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:11.043736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:11.043736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:11.043736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:11.043736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:11.043736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:11.043736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:11.043736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:30:11.043736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:30:11.043736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:30:11.043736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:11.044689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:11.045629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:11.045629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:11.045629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:30:11.045629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:30:11.045629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:30:11.045629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:30:11.045629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:30:11.045629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:30:11.045629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:30:11.045629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:11.049641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 531/1091] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jul 12 13:30:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:11.050581 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6c818-0446-4623-80d8-199f40bd28ef None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:11.148837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6c818-0446-4623-80d8-199f40bd28ef None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:11.181886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8935b41-367e-4289-a3aa-c77161a3d403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:11.182581 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8935b41-367e-4289-a3aa-c77161a3d403 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:11.182869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8935b41-367e-4289-a3aa-c77161a3d403 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:11.183146 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8935b41-367e-4289-a3aa-c77161a3d403 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:11.183969 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8935b41-367e-4289-a3aa-c77161a3d403 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:11.184279 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8935b41-367e-4289-a3aa-c77161a3d403 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:11.185721 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8935b41-367e-4289-a3aa-c77161a3d403 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:11.193332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:11.204664 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 561/1092] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jul 12 13:30:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:11.210499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6218ba4-8f15-48f9-afb3-ee13235440eb None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:11.309879 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6218ba4-8f15-48f9-afb3-ee13235440eb None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:11.343122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5988ab-f747-48ec-b8cd-83c268f9e9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:11.343763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5988ab-f747-48ec-b8cd-83c268f9e9ac None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:11.344054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5988ab-f747-48ec-b8cd-83c268f9e9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:11.344325 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5988ab-f747-48ec-b8cd-83c268f9e9ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:11.371258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:11.411396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5988ab-f747-48ec-b8cd-83c268f9e9ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:11.412781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5988ab-f747-48ec-b8cd-83c268f9e9ac None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:11.513984 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 532/1093] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:30:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:30:11.515268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8935b41-367e-4289-a3aa-c77161a3d403 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:11.612829 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8935b41-367e-4289-a3aa-c77161a3d403 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:11.646240 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e36407-8c28-4791-b3ed-88cca714e97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:11.647030 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e36407-8c28-4791-b3ed-88cca714e97d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:11.647317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e36407-8c28-4791-b3ed-88cca714e97d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:11.647591 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e36407-8c28-4791-b3ed-88cca714e97d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:11.658073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e36407-8c28-4791-b3ed-88cca714e97d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:11.659525 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e36407-8c28-4791-b3ed-88cca714e97d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:11.678149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-77e36407-8c28-4791-b3ed-88cca714e97d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:30:11.678149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:11.678149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:11.678149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:11.678149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:11.678149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:11.678149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:11.678149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:11.678149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:11.678149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:11.678149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:11.678149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:11.678149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:30:11.678149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:30:11.678149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:30:11.679206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:30:11.679206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:11.679206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:11.679206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:11.679206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:11.679206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:11.679206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:11.679206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:11.679206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:11.679206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:11.679206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:11.679206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:11.679206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:11.679206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:11.679206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:11.679206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:11.679206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:11.680116 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:30:11.680116 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:30:11.680116 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:30:11.680116 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:30:11.682109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 562/1094] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:11.684895 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5988ab-f747-48ec-b8cd-83c268f9e9ac None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:11.745421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:11.786363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5988ab-f747-48ec-b8cd-83c268f9e9ac None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:11.818976 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48fabc7-494a-44b6-ae37-77450836a62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:11.819627 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fabc7-494a-44b6-ae37-77450836a62e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:11.819906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fabc7-494a-44b6-ae37-77450836a62e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:11.820181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fabc7-494a-44b6-ae37-77450836a62e None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:11.820887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48fabc7-494a-44b6-ae37-77450836a62e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:11.821193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48fabc7-494a-44b6-ae37-77450836a62e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:11.822615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48fabc7-494a-44b6-ae37-77450836a62e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:11.833326 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 533/1095] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:11.839356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e36407-8c28-4791-b3ed-88cca714e97d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:11.887360 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:11.936151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e36407-8c28-4791-b3ed-88cca714e97d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:11.968229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbc07d-9e7f-4322-8c1e-262efd1de5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:11.969013 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbc07d-9e7f-4322-8c1e-262efd1de5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:11.969292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbc07d-9e7f-4322-8c1e-262efd1de5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:11.969565 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbc07d-9e7f-4322-8c1e-262efd1de5a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:11.976015 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbbc07d-9e7f-4322-8c1e-262efd1de5a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:11.977446 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbbc07d-9e7f-4322-8c1e-262efd1de5a5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:11.991346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-4bbbc07d-9e7f-4322-8c1e-262efd1de5a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:30:11.991346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:11.991346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:11.991346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:11.991346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:11.991346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:11.991346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:11.991346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:11.991346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:11.991346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:11.991346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:11.991346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:11.991346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:30:11.991346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:30:11.991346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:30:11.992382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:30:11.992382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:11.992382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:11.992382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:11.992382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:11.992382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:11.992382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:11.992382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:11.992382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:11.992382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:11.992382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:11.992382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:11.992382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:11.992382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:11.992382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:11.992382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:11.992382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:11.993292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:30:11.993292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:30:11.993292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:30:11.993292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:30:11.993292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:30:11.993292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:30:11.993292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:30:11.993292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:30:11.993292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 563/1096] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:30:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:11.998276 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48fabc7-494a-44b6-ae37-77450836a62e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:12.097974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48fabc7-494a-44b6-ae37-77450836a62e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:12.119947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:12.137724 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8ec9f-b47a-458c-98b6-af78f2be806a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:12.138411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8ec9f-b47a-458c-98b6-af78f2be806a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:12.138692 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8ec9f-b47a-458c-98b6-af78f2be806a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:12.138964 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8ec9f-b47a-458c-98b6-af78f2be806a None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:12.139735 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8ec9f-b47a-458c-98b6-af78f2be806a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:12.140042 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8ec9f-b47a-458c-98b6-af78f2be806a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:12.141460 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8ec9f-b47a-458c-98b6-af78f2be806a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:12.153642 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 534/1097] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:30:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:12.159672 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbc07d-9e7f-4322-8c1e-262efd1de5a5 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:12.264383 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbc07d-9e7f-4322-8c1e-262efd1de5a5 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:12.273183 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:12.298217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e8f12-f07a-47dd-9339-4cce573bede6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:12.298909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e8f12-f07a-47dd-9339-4cce573bede6 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:12.299196 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e8f12-f07a-47dd-9339-4cce573bede6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:12.299476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e8f12-f07a-47dd-9339-4cce573bede6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:12.300299 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e8f12-f07a-47dd-9339-4cce573bede6 None admin] RBAC: Adding query filter params (role.id=19deffe64f224b68880ce003b4af8b2b, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=0d3915cde0d04bcfb9b5ce8c89af1e5b) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:12.300615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e8f12-f07a-47dd-9339-4cce573bede6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:12.302075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e8f12-f07a-47dd-9339-4cce573bede6 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:12.307879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 564/1098] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:30:12 2022] GET /identity/v3/role_assignments?user.id=0d3915cde0d04bcfb9b5ce8c89af1e5b&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=19deffe64f224b68880ce003b4af8b2b => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:12.638602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a1405-13d6-4e84-bded-8d09fc58d23b None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:12.638953 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a1405-13d6-4e84-bded-8d09fc58d23b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:12.639166 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a1405-13d6-4e84-bded-8d09fc58d23b None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:12.640542 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 535/1099] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:12.646104 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeaa443-21e8-4204-a68b-f65502180bb2 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:12.646508 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeaa443-21e8-4204-a68b-f65502180bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:12.646684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeaa443-21e8-4204-a68b-f65502180bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:12.663543 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3eeaa443-21e8-4204-a68b-f65502180bb2 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:12.664733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3eeaa443-21e8-4204-a68b-f65502180bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:12.737205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 565/1100] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:12.761684 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a1405-13d6-4e84-bded-8d09fc58d23b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:12.866242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a1405-13d6-4e84-bded-8d09fc58d23b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:12.901330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6798d9-975d-451c-aa01-60385c3cef60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:12.902153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6798d9-975d-451c-aa01-60385c3cef60 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:12.902441 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6798d9-975d-451c-aa01-60385c3cef60 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:12.902713 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6798d9-975d-451c-aa01-60385c3cef60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:12.913120 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6798d9-975d-451c-aa01-60385c3cef60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:30:12.921372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-5f6798d9-975d-451c-aa01-60385c3cef60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:12.921372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:12.921372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:12.921372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:12.921372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:12.921372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:12.921372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:12.921372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:12.921372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:12.921372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:12.921372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:12.921372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:12.921372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:30:12.921372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:30:12.921372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:30:12.921372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:30:12.922347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:30:12.922347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:30:12.922347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:30:12.922347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:30:12.922347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:12.922347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:12.922347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:12.922347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:12.922347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:12.922347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:12.922347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:12.922347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:12.922347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:12.922347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:12.922347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:12.922347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:12.923214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:12.923214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:12.923214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:12.923214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:12.923214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:30:12.923214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:30:12.923214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:30:12.923214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:30:12.923214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:30:12.923214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:30:12.923214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:12.923214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:12.923214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 536/1101] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:30:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:12.928712 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeaa443-21e8-4204-a68b-f65502180bb2 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:13.029024 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeaa443-21e8-4204-a68b-f65502180bb2 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:13.031245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:13.065326 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfddd8-4722-4795-8e47-e9f33ff320c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:13.066028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfddd8-4722-4795-8e47-e9f33ff320c4 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:13.066317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfddd8-4722-4795-8e47-e9f33ff320c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:13.066602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfddd8-4722-4795-8e47-e9f33ff320c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:13.067349 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdfddd8-4722-4795-8e47-e9f33ff320c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:13.067662 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdfddd8-4722-4795-8e47-e9f33ff320c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:13.069048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdfddd8-4722-4795-8e47-e9f33ff320c4 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:13.077952 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 566/1102] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:30:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:13.083806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6798d9-975d-451c-aa01-60385c3cef60 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:13.128340 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:13.189671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6798d9-975d-451c-aa01-60385c3cef60 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:13.222056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05d710-9c6f-4a12-83be-29e70772ae28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:13.222917 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05d710-9c6f-4a12-83be-29e70772ae28 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:13.223202 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05d710-9c6f-4a12-83be-29e70772ae28 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:13.223479 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05d710-9c6f-4a12-83be-29e70772ae28 None admin] PATH_INFO: `/v3/users/50a99119d9374da79b917ceb19264ccf` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:13.224661 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05d710-9c6f-4a12-83be-29e70772ae28 None admin] RBAC: Authorizing `identity:get_user(user_id=50a99119d9374da79b917ceb19264ccf)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:13.226119 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05d710-9c6f-4a12-83be-29e70772ae28 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:13.227763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 537/1103] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:30:13 2022] GET /identity/v3/users/50a99119d9374da79b917ceb19264ccf => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:13.234166 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfddd8-4722-4795-8e47-e9f33ff320c4 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:13.330228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfddd8-4722-4795-8e47-e9f33ff320c4 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:13.362922 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06517915-b73e-437f-a77f-bd175fc0400a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:13.363560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06517915-b73e-437f-a77f-bd175fc0400a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:13.363844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06517915-b73e-437f-a77f-bd175fc0400a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:13.364116 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06517915-b73e-437f-a77f-bd175fc0400a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:13.393460 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:13.430047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06517915-b73e-437f-a77f-bd175fc0400a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:13.431528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06517915-b73e-437f-a77f-bd175fc0400a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:13.530433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 567/1104] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:30:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:30:13.536246 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05d710-9c6f-4a12-83be-29e70772ae28 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:13.633060 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05d710-9c6f-4a12-83be-29e70772ae28 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:13.640337 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:13.667071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d40a54-70ce-4f64-9624-85f4b9f63df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:13.667887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d40a54-70ce-4f64-9624-85f4b9f63df4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:13.668173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d40a54-70ce-4f64-9624-85f4b9f63df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:13.668449 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d40a54-70ce-4f64-9624-85f4b9f63df4 None admin] PATH_INFO: `/v3/projects/97655558670b4282aec6d5144211f188` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:13.669734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d40a54-70ce-4f64-9624-85f4b9f63df4 None admin] RBAC: Authorizing `identity:get_project(project_id=97655558670b4282aec6d5144211f188)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:13.671226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d40a54-70ce-4f64-9624-85f4b9f63df4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:13.672982 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 538/1105] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:30:13 2022] GET /identity/v3/projects/97655558670b4282aec6d5144211f188 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:13.679087 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06517915-b73e-437f-a77f-bd175fc0400a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:13.738140 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:13.775498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06517915-b73e-437f-a77f-bd175fc0400a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:13.807610 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4019a65-d1fd-4001-b0e0-e3f3d21d6bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:13.808186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4019a65-d1fd-4001-b0e0-e3f3d21d6bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:13.808467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4019a65-d1fd-4001-b0e0-e3f3d21d6bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:13.808742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4019a65-d1fd-4001-b0e0-e3f3d21d6bb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:13.809550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4019a65-d1fd-4001-b0e0-e3f3d21d6bb5 None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=97655558670b4282aec6d5144211f188, user.id=50a99119d9374da79b917ceb19264ccf) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:13.809858 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4019a65-d1fd-4001-b0e0-e3f3d21d6bb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:13.811312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4019a65-d1fd-4001-b0e0-e3f3d21d6bb5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:13.816906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 568/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:30:13 2022] GET /identity/v3/role_assignments?user.id=50a99119d9374da79b917ceb19264ccf&scope.project.id=97655558670b4282aec6d5144211f188&role.id=864a6966630544e8bd68bcafba905c6f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:15.874724 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4f678-dbfc-44af-9ce9-fd26f0b48976 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:15.875238 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4f678-dbfc-44af-9ce9-fd26f0b48976 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:15.875338 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4f678-dbfc-44af-9ce9-fd26f0b48976 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:15.876896 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 539/1107] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:15.882184 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b38e7f-1d4d-47d0-aaf2-566c35bad269 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:15.882581 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b38e7f-1d4d-47d0-aaf2-566c35bad269 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:15.882738 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b38e7f-1d4d-47d0-aaf2-566c35bad269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:15.898964 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-10b38e7f-1d4d-47d0-aaf2-566c35bad269 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:15.900134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-10b38e7f-1d4d-47d0-aaf2-566c35bad269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:15.964529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 569/1108] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:15.989134 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4f678-dbfc-44af-9ce9-fd26f0b48976 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:16.087587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4f678-dbfc-44af-9ce9-fd26f0b48976 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:16.121939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71469321-0ec4-4735-872f-d69d48bba8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:16.122758 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71469321-0ec4-4735-872f-d69d48bba8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:16.123049 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71469321-0ec4-4735-872f-d69d48bba8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:16.123327 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71469321-0ec4-4735-872f-d69d48bba8f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:16.124499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71469321-0ec4-4735-872f-d69d48bba8f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:16.126228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71469321-0ec4-4735-872f-d69d48bba8f6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:16.127873 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 540/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:30:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:16.133611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b38e7f-1d4d-47d0-aaf2-566c35bad269 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:16.190484 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:16.231367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b38e7f-1d4d-47d0-aaf2-566c35bad269 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:16.264193 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86172f7c-f59b-49aa-9f4c-1343e4541ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:16.264856 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86172f7c-f59b-49aa-9f4c-1343e4541ae8 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:16.265141 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86172f7c-f59b-49aa-9f4c-1343e4541ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:16.265420 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86172f7c-f59b-49aa-9f4c-1343e4541ae8 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:16.266153 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86172f7c-f59b-49aa-9f4c-1343e4541ae8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:16.267600 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86172f7c-f59b-49aa-9f4c-1343e4541ae8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:16.287170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 570/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:30:16 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:30:18.329149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41427c-bf13-4d09-be4b-0ec9303a2a51 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:18.329689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41427c-bf13-4d09-be4b-0ec9303a2a51 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:18.329689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41427c-bf13-4d09-be4b-0ec9303a2a51 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:18.331111 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 541/1111] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:18.336156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9aeef0-d949-4037-ac97-3918c0832fc8 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:18.336548 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9aeef0-d949-4037-ac97-3918c0832fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:18.336716 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9aeef0-d949-4037-ac97-3918c0832fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:18.354014 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-db9aeef0-d949-4037-ac97-3918c0832fc8 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:18.355207 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-db9aeef0-d949-4037-ac97-3918c0832fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:18.422296 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 571/1112] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:18.445804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41427c-bf13-4d09-be4b-0ec9303a2a51 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:18.489012 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:18.546139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41427c-bf13-4d09-be4b-0ec9303a2a51 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:18.582214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802528cf-bd6c-4649-9ca7-0a4540c74ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:18.582986 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802528cf-bd6c-4649-9ca7-0a4540c74ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:18.583267 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802528cf-bd6c-4649-9ca7-0a4540c74ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:18.583540 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802528cf-bd6c-4649-9ca7-0a4540c74ea1 None admin] PATH_INFO: `/v3/roles/dbe30950eded4c9ab2bcc0f7545b9bae` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:18.585032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802528cf-bd6c-4649-9ca7-0a4540c74ea1 None admin] RBAC: Authorizing `identity:get_role(role_id=dbe30950eded4c9ab2bcc0f7545b9bae)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:18.586438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802528cf-bd6c-4649-9ca7-0a4540c74ea1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:18.587842 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 542/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:30:18 2022] GET /identity/v3/roles/dbe30950eded4c9ab2bcc0f7545b9bae => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:18.593684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9aeef0-d949-4037-ac97-3918c0832fc8 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:18.638381 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:18.691555 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9aeef0-d949-4037-ac97-3918c0832fc8 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:18.724804 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db261d-c0a0-49b8-a551-0f527acfb917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:18.725676 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db261d-c0a0-49b8-a551-0f527acfb917 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:18.725990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db261d-c0a0-49b8-a551-0f527acfb917 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:18.726264 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db261d-c0a0-49b8-a551-0f527acfb917 None admin] PATH_INFO: `/v3/users/53eae551638147d996ca2673aa0964e8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:18.739076 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db261d-c0a0-49b8-a551-0f527acfb917 None admin] RBAC: Authorizing `identity:get_user(user_id=53eae551638147d996ca2673aa0964e8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:18.740531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db261d-c0a0-49b8-a551-0f527acfb917 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:18.742237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 572/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:30:18 2022] GET /identity/v3/users/53eae551638147d996ca2673aa0964e8 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:18.748500 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802528cf-bd6c-4649-9ca7-0a4540c74ea1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:18.844296 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802528cf-bd6c-4649-9ca7-0a4540c74ea1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:18.855005 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:18.879151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04878e2-ea2e-481a-bc38-9b94194c4ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:18.879795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04878e2-ea2e-481a-bc38-9b94194c4ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:18.880084 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04878e2-ea2e-481a-bc38-9b94194c4ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:18.880360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04878e2-ea2e-481a-bc38-9b94194c4ec2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:18.944808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04878e2-ea2e-481a-bc38-9b94194c4ec2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:18.946277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04878e2-ea2e-481a-bc38-9b94194c4ec2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:19.043237 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 543/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:30:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:30:19.049188 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db261d-c0a0-49b8-a551-0f527acfb917 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:19.145405 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db261d-c0a0-49b8-a551-0f527acfb917 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:19.146940 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:19.180314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf2011-03a3-41b1-9271-fd0bc4978fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:19.181189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf2011-03a3-41b1-9271-fd0bc4978fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:19.181472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf2011-03a3-41b1-9271-fd0bc4978fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:19.181744 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf2011-03a3-41b1-9271-fd0bc4978fb3 None admin] PATH_INFO: `/v3/projects/97655558670b4282aec6d5144211f188` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:19.182945 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf2011-03a3-41b1-9271-fd0bc4978fb3 None admin] RBAC: Authorizing `identity:get_project(project_id=97655558670b4282aec6d5144211f188)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:19.184418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf2011-03a3-41b1-9271-fd0bc4978fb3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:19.186107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 573/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:30:19 2022] GET /identity/v3/projects/97655558670b4282aec6d5144211f188 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:19.192246 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04878e2-ea2e-481a-bc38-9b94194c4ec2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:19.203422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:19.286674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04878e2-ea2e-481a-bc38-9b94194c4ec2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:19.319587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c43d82-2766-4313-8eb1-b60bfbcca7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:19.320231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c43d82-2766-4313-8eb1-b60bfbcca7ed None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:19.320515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c43d82-2766-4313-8eb1-b60bfbcca7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:19.320793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c43d82-2766-4313-8eb1-b60bfbcca7ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:19.321607 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c43d82-2766-4313-8eb1-b60bfbcca7ed None admin] RBAC: Adding query filter params (role.id=dbe30950eded4c9ab2bcc0f7545b9bae, scope.project.id=97655558670b4282aec6d5144211f188, user.id=53eae551638147d996ca2673aa0964e8) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:19.321922 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c43d82-2766-4313-8eb1-b60bfbcca7ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:19.323377 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c43d82-2766-4313-8eb1-b60bfbcca7ed None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:19.328686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 544/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:30:19 2022] GET /identity/v3/role_assignments?user.id=53eae551638147d996ca2673aa0964e8&scope.project.id=97655558670b4282aec6d5144211f188&role.id=dbe30950eded4c9ab2bcc0f7545b9bae => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:21.323234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fc022-3a8e-4b4f-a7c5-b57224d23a33 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:21.323789 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fc022-3a8e-4b4f-a7c5-b57224d23a33 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:21.323789 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fc022-3a8e-4b4f-a7c5-b57224d23a33 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:21.325154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 574/1118] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:21.330428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7753454-d703-4e6e-b720-b832e8cb060f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:21.330858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7753454-d703-4e6e-b720-b832e8cb060f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:21.331008 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7753454-d703-4e6e-b720-b832e8cb060f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:21.347764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a7753454-d703-4e6e-b720-b832e8cb060f None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:21.348934 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a7753454-d703-4e6e-b720-b832e8cb060f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:21.413233 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 545/1119] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:21.435507 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fc022-3a8e-4b4f-a7c5-b57224d23a33 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:21.534330 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fc022-3a8e-4b4f-a7c5-b57224d23a33 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:21.569291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1c23b-b8b6-436d-851f-f5a80c384f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:21.570178 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1c23b-b8b6-436d-851f-f5a80c384f44 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:21.570466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1c23b-b8b6-436d-851f-f5a80c384f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:21.570743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1c23b-b8b6-436d-851f-f5a80c384f44 None admin] PATH_INFO: `/v3/roles/dbe30950eded4c9ab2bcc0f7545b9bae` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:21.572749 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1c23b-b8b6-436d-851f-f5a80c384f44 None admin] RBAC: Authorizing `identity:get_role(role_id=dbe30950eded4c9ab2bcc0f7545b9bae)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:21.574208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1c23b-b8b6-436d-851f-f5a80c384f44 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:21.575533 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 575/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:30:21 2022] GET /identity/v3/roles/dbe30950eded4c9ab2bcc0f7545b9bae => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:21.581398 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7753454-d703-4e6e-b720-b832e8cb060f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:21.584554 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:21.681067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7753454-d703-4e6e-b720-b832e8cb060f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:21.715372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e08be-5114-4170-967b-342e849c7e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:21.716237 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e08be-5114-4170-967b-342e849c7e68 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:21.716516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e08be-5114-4170-967b-342e849c7e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:21.716791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e08be-5114-4170-967b-342e849c7e68 None admin] PATH_INFO: `/v3/users/53eae551638147d996ca2673aa0964e8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:21.717999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e08be-5114-4170-967b-342e849c7e68 None admin] RBAC: Authorizing `identity:get_user(user_id=53eae551638147d996ca2673aa0964e8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:21.719433 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e08be-5114-4170-967b-342e849c7e68 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:21.721080 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 546/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:30:21 2022] GET /identity/v3/users/53eae551638147d996ca2673aa0964e8 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:21.726987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1c23b-b8b6-436d-851f-f5a80c384f44 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:21.777268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:21.846517 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1c23b-b8b6-436d-851f-f5a80c384f44 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:21.880859 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94937faa-ab0f-4828-aacc-f8df6c64d10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:21.881725 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94937faa-ab0f-4828-aacc-f8df6c64d10b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:21.882044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94937faa-ab0f-4828-aacc-f8df6c64d10b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:21.882324 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94937faa-ab0f-4828-aacc-f8df6c64d10b None admin] PATH_INFO: `/v3/projects/97655558670b4282aec6d5144211f188` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:21.883498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94937faa-ab0f-4828-aacc-f8df6c64d10b None admin] RBAC: Authorizing `identity:get_project(project_id=97655558670b4282aec6d5144211f188)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:21.884937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94937faa-ab0f-4828-aacc-f8df6c64d10b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:21.886655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 576/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:30:21 2022] GET /identity/v3/projects/97655558670b4282aec6d5144211f188 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:21.892587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e08be-5114-4170-967b-342e849c7e68 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:21.960019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:21.987992 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e08be-5114-4170-967b-342e849c7e68 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:22.020689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c9e0f-75af-460e-9de3-b87817ec05f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:22.021558 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c9e0f-75af-460e-9de3-b87817ec05f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:22.021833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c9e0f-75af-460e-9de3-b87817ec05f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:22.022207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c9e0f-75af-460e-9de3-b87817ec05f7 None admin] PATH_INFO: `/v3/projects/97655558670b4282aec6d5144211f188/users/53eae551638147d996ca2673aa0964e8/roles/dbe30950eded4c9ab2bcc0f7545b9bae` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:22.024376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c9e0f-75af-460e-9de3-b87817ec05f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=97655558670b4282aec6d5144211f188, user_id=53eae551638147d996ca2673aa0964e8, role_id=dbe30950eded4c9ab2bcc0f7545b9bae)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:22.025792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c9e0f-75af-460e-9de3-b87817ec05f7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:22.038422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 547/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:30:21 2022] PUT /identity/v3/projects/97655558670b4282aec6d5144211f188/users/53eae551638147d996ca2673aa0964e8/roles/dbe30950eded4c9ab2bcc0f7545b9bae => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:30:24.067144 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa9ebe-cf50-4295-b513-c7b6149716d6 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:24.067688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa9ebe-cf50-4295-b513-c7b6149716d6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:24.067688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa9ebe-cf50-4295-b513-c7b6149716d6 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:24.069034 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 577/1124] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:24.074905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836412b-5f2c-42fd-94fb-ccd7e3f5a17c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:24.075327 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836412b-5f2c-42fd-94fb-ccd7e3f5a17c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:24.075462 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836412b-5f2c-42fd-94fb-ccd7e3f5a17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:24.092348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f836412b-5f2c-42fd-94fb-ccd7e3f5a17c None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:24.093539 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f836412b-5f2c-42fd-94fb-ccd7e3f5a17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:24.158184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 548/1125] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:24.181382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa9ebe-cf50-4295-b513-c7b6149716d6 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:24.280638 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa9ebe-cf50-4295-b513-c7b6149716d6 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:24.287851 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:24.313749 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2904bb0-0d39-422d-a377-3585880e8662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:24.314625 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2904bb0-0d39-422d-a377-3585880e8662 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:24.314904 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2904bb0-0d39-422d-a377-3585880e8662 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:24.315170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2904bb0-0d39-422d-a377-3585880e8662 None admin] PATH_INFO: `/v3/roles/dbe30950eded4c9ab2bcc0f7545b9bae` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:24.316624 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2904bb0-0d39-422d-a377-3585880e8662 None admin] RBAC: Authorizing `identity:get_role(role_id=dbe30950eded4c9ab2bcc0f7545b9bae)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:24.318022 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2904bb0-0d39-422d-a377-3585880e8662 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:24.319375 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 578/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:30:24 2022] GET /identity/v3/roles/dbe30950eded4c9ab2bcc0f7545b9bae => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:24.325806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836412b-5f2c-42fd-94fb-ccd7e3f5a17c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:24.385477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:24.423741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836412b-5f2c-42fd-94fb-ccd7e3f5a17c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:24.457395 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686df5c-9cb3-4695-b7a1-e864a8dc8320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:24.458290 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686df5c-9cb3-4695-b7a1-e864a8dc8320 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:24.458577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686df5c-9cb3-4695-b7a1-e864a8dc8320 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:24.458854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686df5c-9cb3-4695-b7a1-e864a8dc8320 None admin] PATH_INFO: `/v3/users/53eae551638147d996ca2673aa0964e8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:24.460040 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686df5c-9cb3-4695-b7a1-e864a8dc8320 None admin] RBAC: Authorizing `identity:get_user(user_id=53eae551638147d996ca2673aa0964e8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:24.461473 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686df5c-9cb3-4695-b7a1-e864a8dc8320 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:24.463153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 549/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:30:24 2022] GET /identity/v3/users/53eae551638147d996ca2673aa0964e8 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:24.468848 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2904bb0-0d39-422d-a377-3585880e8662 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:24.565702 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2904bb0-0d39-422d-a377-3585880e8662 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:24.597845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf1a1d-da63-4924-aea3-34ee08363f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:24.598506 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf1a1d-da63-4924-aea3-34ee08363f97 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:24.598785 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf1a1d-da63-4924-aea3-34ee08363f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:24.599049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf1a1d-da63-4924-aea3-34ee08363f97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:24.642450 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:24.666906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf1a1d-da63-4924-aea3-34ee08363f97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:24.668371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf1a1d-da63-4924-aea3-34ee08363f97 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:24.764308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 579/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:30:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:30:24.770147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686df5c-9cb3-4695-b7a1-e864a8dc8320 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:24.867510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686df5c-9cb3-4695-b7a1-e864a8dc8320 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:24.895576 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:24.901660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212df86-398a-4ade-a8d9-f2c0830aed1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:24.902613 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212df86-398a-4ade-a8d9-f2c0830aed1f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:24.902895 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212df86-398a-4ade-a8d9-f2c0830aed1f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:24.903168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212df86-398a-4ade-a8d9-f2c0830aed1f None admin] PATH_INFO: `/v3/projects/97655558670b4282aec6d5144211f188` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:24.904332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212df86-398a-4ade-a8d9-f2c0830aed1f None admin] RBAC: Authorizing `identity:get_project(project_id=97655558670b4282aec6d5144211f188)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:24.905758 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212df86-398a-4ade-a8d9-f2c0830aed1f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:24.907492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 550/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:30:24 2022] GET /identity/v3/projects/97655558670b4282aec6d5144211f188 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:24.913535 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf1a1d-da63-4924-aea3-34ee08363f97 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:24.985495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:25.010732 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf1a1d-da63-4924-aea3-34ee08363f97 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:25.043268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16234d8a-6990-4d0a-a1a0-dda9520a4f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:25.043853 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16234d8a-6990-4d0a-a1a0-dda9520a4f1d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:25.044141 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16234d8a-6990-4d0a-a1a0-dda9520a4f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:25.044422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16234d8a-6990-4d0a-a1a0-dda9520a4f1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:25.045238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16234d8a-6990-4d0a-a1a0-dda9520a4f1d None admin] RBAC: Adding query filter params (role.id=dbe30950eded4c9ab2bcc0f7545b9bae, scope.project.id=97655558670b4282aec6d5144211f188, user.id=53eae551638147d996ca2673aa0964e8) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:25.045556 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16234d8a-6990-4d0a-a1a0-dda9520a4f1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:25.047030 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16234d8a-6990-4d0a-a1a0-dda9520a4f1d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:25.052713 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 580/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:30:24 2022] GET /identity/v3/role_assignments?user.id=53eae551638147d996ca2673aa0964e8&scope.project.id=97655558670b4282aec6d5144211f188&role.id=dbe30950eded4c9ab2bcc0f7545b9bae => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:27.088121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce23076-2007-46e0-9242-a025af0d27a9 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:27.088640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce23076-2007-46e0-9242-a025af0d27a9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:27.088745 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce23076-2007-46e0-9242-a025af0d27a9 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:27.090079 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 551/1131] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:27.095132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c4092-38c2-46b8-abf5-d60351cfa66f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:27.095544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c4092-38c2-46b8-abf5-d60351cfa66f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:27.095632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c4092-38c2-46b8-abf5-d60351cfa66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:27.112626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4b5c4092-38c2-46b8-abf5-d60351cfa66f None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:27.113816 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4b5c4092-38c2-46b8-abf5-d60351cfa66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:27.177942 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 581/1132] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:27.201340 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce23076-2007-46e0-9242-a025af0d27a9 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:27.297229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce23076-2007-46e0-9242-a025af0d27a9 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:27.329384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811cdb26-0e6d-4dd6-bb9c-12741c3f4cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:27.330184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cdb26-0e6d-4dd6-bb9c-12741c3f4cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:27.330464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cdb26-0e6d-4dd6-bb9c-12741c3f4cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:27.330738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cdb26-0e6d-4dd6-bb9c-12741c3f4cf3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:27.331914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811cdb26-0e6d-4dd6-bb9c-12741c3f4cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:27.333318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811cdb26-0e6d-4dd6-bb9c-12741c3f4cf3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:27.335011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 552/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:30:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:27.340762 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c4092-38c2-46b8-abf5-d60351cfa66f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:27.405562 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:27.438549 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c4092-38c2-46b8-abf5-d60351cfa66f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:27.470936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d44041-0929-4fcf-b088-de2f257aa58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:27.471584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d44041-0929-4fcf-b088-de2f257aa58b None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:27.471869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d44041-0929-4fcf-b088-de2f257aa58b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:27.472146 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d44041-0929-4fcf-b088-de2f257aa58b None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:27.472846 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d44041-0929-4fcf-b088-de2f257aa58b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:27.474311 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d44041-0929-4fcf-b088-de2f257aa58b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:27.498900 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 582/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jul 12 13:30:27 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:30:29.544916 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fba46-5515-4e15-908b-47dda072dc6f None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:29.545448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fba46-5515-4e15-908b-47dda072dc6f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:29.545448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fba46-5515-4e15-908b-47dda072dc6f None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:29.546883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 553/1135] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:29.552126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a2fed-67ad-4fca-a713-c806eabd2af9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:29.552548 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a2fed-67ad-4fca-a713-c806eabd2af9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:29.552681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a2fed-67ad-4fca-a713-c806eabd2af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:29.570375 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1a3a2fed-67ad-4fca-a713-c806eabd2af9 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:29.572086 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1a3a2fed-67ad-4fca-a713-c806eabd2af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:29.638177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 583/1136] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:29.662922 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fba46-5515-4e15-908b-47dda072dc6f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:29.727892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:29.764529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fba46-5515-4e15-908b-47dda072dc6f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:29.796711 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bf6a7-84a8-4b71-b36b-2ecaafd4f261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:29.797423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bf6a7-84a8-4b71-b36b-2ecaafd4f261 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:29.797702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bf6a7-84a8-4b71-b36b-2ecaafd4f261 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:29.798001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bf6a7-84a8-4b71-b36b-2ecaafd4f261 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:29.799175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0bf6a7-84a8-4b71-b36b-2ecaafd4f261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:29.800602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0bf6a7-84a8-4b71-b36b-2ecaafd4f261 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:29.802245 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 554/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:30:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:29.808511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a2fed-67ad-4fca-a713-c806eabd2af9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:29.856949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:29.906640 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a2fed-67ad-4fca-a713-c806eabd2af9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:29.939183 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f6964-9ce7-40de-ad56-ae6c5998b4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:29.939846 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6964-9ce7-40de-ad56-ae6c5998b4f3 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:29.940128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6964-9ce7-40de-ad56-ae6c5998b4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:29.940402 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6964-9ce7-40de-ad56-ae6c5998b4f3 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:29.941097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f6964-9ce7-40de-ad56-ae6c5998b4f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:29.942582 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f6964-9ce7-40de-ad56-ae6c5998b4f3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:29.961090 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 584/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:30:29 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:30:32.054924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f6aab-2e07-4c33-a34c-da01ce0fddf5 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:32.056003 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f6aab-2e07-4c33-a34c-da01ce0fddf5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:32.056003 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f6aab-2e07-4c33-a34c-da01ce0fddf5 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:32.057340 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 555/1139] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:32.062397 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7c51d-c313-497d-a249-4e94315e5b03 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:32.062812 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7c51d-c313-497d-a249-4e94315e5b03 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:32.062954 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7c51d-c313-497d-a249-4e94315e5b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:32.080246 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-89c7c51d-c313-497d-a249-4e94315e5b03 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:32.081421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-89c7c51d-c313-497d-a249-4e94315e5b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:32.146609 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 585/1140] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:32.169566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f6aab-2e07-4c33-a34c-da01ce0fddf5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:32.270497 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f6aab-2e07-4c33-a34c-da01ce0fddf5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:32.303101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19151c-1327-43fc-b679-0fc43db9d4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:32.303883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19151c-1327-43fc-b679-0fc43db9d4ea None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:32.304166 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19151c-1327-43fc-b679-0fc43db9d4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:32.304437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19151c-1327-43fc-b679-0fc43db9d4ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:32.306912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:32.316099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19151c-1327-43fc-b679-0fc43db9d4ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:30:32.323783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-7b19151c-1327-43fc-b679-0fc43db9d4ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:32.323783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:32.323783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:32.323783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:32.323783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:32.323783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:32.323783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:32.323783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:32.323783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:32.323783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:32.323783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:32.323783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:32.323783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:30:32.323783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:30:32.323783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:30:32.323783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:30:32.324979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:30:32.324979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:30:32.324979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:30:32.324979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:30:32.324979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:32.324979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:32.324979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:32.324979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:32.324979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:32.324979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:32.324979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:32.324979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:32.324979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:32.324979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:32.324979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:32.324979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:32.325951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:32.325951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:32.325951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:32.325951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:32.325951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:30:32.325951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:30:32.325951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:30:32.325951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:30:32.325951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:30:32.325951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:30:32.325951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:32.325951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:32.325951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 556/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:30:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:32.331060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7c51d-c313-497d-a249-4e94315e5b03 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:32.364848 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:32.428406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7c51d-c313-497d-a249-4e94315e5b03 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:32.461766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d46f8-fc0b-464f-b7e8-a8e6eabd59ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:32.462460 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d46f8-fc0b-464f-b7e8-a8e6eabd59ab None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:32.462747 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d46f8-fc0b-464f-b7e8-a8e6eabd59ab None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:32.463027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d46f8-fc0b-464f-b7e8-a8e6eabd59ab None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:32.463770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d46f8-fc0b-464f-b7e8-a8e6eabd59ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:32.464082 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d46f8-fc0b-464f-b7e8-a8e6eabd59ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:32.465473 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d46f8-fc0b-464f-b7e8-a8e6eabd59ab None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:32.473402 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 586/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:30:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:32.479637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19151c-1327-43fc-b679-0fc43db9d4ea None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:32.577439 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19151c-1327-43fc-b679-0fc43db9d4ea None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:32.609728 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace52a5-cad1-45f3-9b1c-6ffea44505b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:32.610606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace52a5-cad1-45f3-9b1c-6ffea44505b4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:32.610885 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace52a5-cad1-45f3-9b1c-6ffea44505b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:32.611153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace52a5-cad1-45f3-9b1c-6ffea44505b4 None admin] PATH_INFO: `/v3/users/8b7d729d97634866b99714068e8f5c24` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:32.624370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace52a5-cad1-45f3-9b1c-6ffea44505b4 None admin] RBAC: Authorizing `identity:get_user(user_id=8b7d729d97634866b99714068e8f5c24)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:32.626092 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace52a5-cad1-45f3-9b1c-6ffea44505b4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:32.627741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 557/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:30:32 2022] GET /identity/v3/users/8b7d729d97634866b99714068e8f5c24 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:32.633479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d46f8-fc0b-464f-b7e8-a8e6eabd59ab None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:32.728862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:32.732996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d46f8-fc0b-464f-b7e8-a8e6eabd59ab None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:32.765375 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e895535-f749-4109-bf2d-d2867899b18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:32.766036 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e895535-f749-4109-bf2d-d2867899b18e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:32.766320 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e895535-f749-4109-bf2d-d2867899b18e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:32.766593 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e895535-f749-4109-bf2d-d2867899b18e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:32.830743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e895535-f749-4109-bf2d-d2867899b18e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:32.832190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e895535-f749-4109-bf2d-d2867899b18e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:32.924035 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:32.929876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 587/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:30:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:30:32.935542 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace52a5-cad1-45f3-9b1c-6ffea44505b4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:32.987238 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:33.032955 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace52a5-cad1-45f3-9b1c-6ffea44505b4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:33.064776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d8e7a-98a4-4317-8ac1-a29bdbf6e717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:33.065621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d8e7a-98a4-4317-8ac1-a29bdbf6e717 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:33.065897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d8e7a-98a4-4317-8ac1-a29bdbf6e717 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:33.066192 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d8e7a-98a4-4317-8ac1-a29bdbf6e717 None admin] PATH_INFO: `/v3/projects/2bf135d016d2468ea74f657e5ba96ce0` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:33.067349 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d8e7a-98a4-4317-8ac1-a29bdbf6e717 None admin] RBAC: Authorizing `identity:get_project(project_id=2bf135d016d2468ea74f657e5ba96ce0)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:33.068761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d8e7a-98a4-4317-8ac1-a29bdbf6e717 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:33.070480 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 558/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:30:32 2022] GET /identity/v3/projects/2bf135d016d2468ea74f657e5ba96ce0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:33.077254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e895535-f749-4109-bf2d-d2867899b18e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:33.175605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e895535-f749-4109-bf2d-d2867899b18e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:33.218352 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61484e84-c350-4a7b-9052-ae0d3b326b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:33.219307 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61484e84-c350-4a7b-9052-ae0d3b326b6d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:33.219728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61484e84-c350-4a7b-9052-ae0d3b326b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:33.220148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61484e84-c350-4a7b-9052-ae0d3b326b6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:33.221464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61484e84-c350-4a7b-9052-ae0d3b326b6d None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=2bf135d016d2468ea74f657e5ba96ce0, user.id=8b7d729d97634866b99714068e8f5c24) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:33.221941 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61484e84-c350-4a7b-9052-ae0d3b326b6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:33.224229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61484e84-c350-4a7b-9052-ae0d3b326b6d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:33.230362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 588/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:30:33 2022] GET /identity/v3/role_assignments?user.id=8b7d729d97634866b99714068e8f5c24&scope.project.id=2bf135d016d2468ea74f657e5ba96ce0&role.id=864a6966630544e8bd68bcafba905c6f => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:35.214828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b3c7a-8734-469a-a530-724a4815301a None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:35.215392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b3c7a-8734-469a-a530-724a4815301a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:35.215392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b3c7a-8734-469a-a530-724a4815301a None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:35.216772 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 559/1147] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:35.222219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa7ac6-0284-4637-9108-2d40dbb8515f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:35.222647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa7ac6-0284-4637-9108-2d40dbb8515f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:35.222803 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa7ac6-0284-4637-9108-2d40dbb8515f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:35.239005 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-64fa7ac6-0284-4637-9108-2d40dbb8515f None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:35.240177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-64fa7ac6-0284-4637-9108-2d40dbb8515f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:35.294685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:35.305045 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 589/1148] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:35.327252 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b3c7a-8734-469a-a530-724a4815301a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:35.424056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b3c7a-8734-469a-a530-724a4815301a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:35.443670 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:35.456906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcb66a-0d56-432a-87f0-33ff8831d264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:35.457683 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcb66a-0d56-432a-87f0-33ff8831d264 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:35.457984 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcb66a-0d56-432a-87f0-33ff8831d264 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:35.458260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcb66a-0d56-432a-87f0-33ff8831d264 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:35.467913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edcb66a-0d56-432a-87f0-33ff8831d264 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:30:35.475515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-3edcb66a-0d56-432a-87f0-33ff8831d264 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:35.475515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:35.475515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:35.475515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:35.475515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:35.475515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:35.475515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:35.475515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:35.475515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:35.475515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:35.475515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:35.475515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:35.475515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:30:35.475515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:30:35.475515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:30:35.475515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:30:35.476632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:30:35.476632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:30:35.476632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:30:35.476632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:30:35.476632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:35.476632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:35.476632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:35.476632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:35.476632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:35.476632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:35.476632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:35.476632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:35.476632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:35.476632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:35.476632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:35.476632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:35.478204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:35.478204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:35.478204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:35.478204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:35.478204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:30:35.478204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:30:35.478204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:30:35.478204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:30:35.478204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:30:35.478204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:30:35.478204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:35.478204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:35.478204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 560/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:30:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:35.483273 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa7ac6-0284-4637-9108-2d40dbb8515f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:35.581292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa7ac6-0284-4637-9108-2d40dbb8515f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:35.615335 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b20c7-90db-44f9-ab9a-2b4276dbf3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:35.616000 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b20c7-90db-44f9-ab9a-2b4276dbf3ed None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:35.616290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b20c7-90db-44f9-ab9a-2b4276dbf3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:35.616572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b20c7-90db-44f9-ab9a-2b4276dbf3ed None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:35.617310 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b20c7-90db-44f9-ab9a-2b4276dbf3ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:35.618206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b20c7-90db-44f9-ab9a-2b4276dbf3ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:35.619705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b20c7-90db-44f9-ab9a-2b4276dbf3ed None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:35.629232 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 590/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:30:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:35.634999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcb66a-0d56-432a-87f0-33ff8831d264 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:35.738307 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcb66a-0d56-432a-87f0-33ff8831d264 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:35.770324 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5acd0b-575a-4628-9655-da7bfc8b9c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:35.771182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5acd0b-575a-4628-9655-da7bfc8b9c27 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:35.771461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5acd0b-575a-4628-9655-da7bfc8b9c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:35.771731 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5acd0b-575a-4628-9655-da7bfc8b9c27 None admin] PATH_INFO: `/v3/users/8b7d729d97634866b99714068e8f5c24` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:35.772897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5acd0b-575a-4628-9655-da7bfc8b9c27 None admin] RBAC: Authorizing `identity:get_user(user_id=8b7d729d97634866b99714068e8f5c24)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:35.774862 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5acd0b-575a-4628-9655-da7bfc8b9c27 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:35.776482 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 561/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:30:35 2022] GET /identity/v3/users/8b7d729d97634866b99714068e8f5c24 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:35.782599 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b20c7-90db-44f9-ab9a-2b4276dbf3ed None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:35.836009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:35.881459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b20c7-90db-44f9-ab9a-2b4276dbf3ed None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:35.914039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525553c-6ca6-4696-9be2-fd5606574146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:35.915482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525553c-6ca6-4696-9be2-fd5606574146 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:35.915766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525553c-6ca6-4696-9be2-fd5606574146 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:35.916045 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525553c-6ca6-4696-9be2-fd5606574146 None admin] PATH_INFO: `/v3/projects/2bf135d016d2468ea74f657e5ba96ce0` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:35.917223 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525553c-6ca6-4696-9be2-fd5606574146 None admin] RBAC: Authorizing `identity:get_project(project_id=2bf135d016d2468ea74f657e5ba96ce0)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:35.918696 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525553c-6ca6-4696-9be2-fd5606574146 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:35.920390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 591/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:30:35 2022] GET /identity/v3/projects/2bf135d016d2468ea74f657e5ba96ce0 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:35.926333 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5acd0b-575a-4628-9655-da7bfc8b9c27 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:35.975204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:36.031349 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5acd0b-575a-4628-9655-da7bfc8b9c27 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:36.063933 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4126de-a1d7-463f-a923-3bcff9f703e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:36.064813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4126de-a1d7-463f-a923-3bcff9f703e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:36.065089 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4126de-a1d7-463f-a923-3bcff9f703e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:36.065364 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4126de-a1d7-463f-a923-3bcff9f703e8 None admin] PATH_INFO: `/v3/projects/2bf135d016d2468ea74f657e5ba96ce0/users/8b7d729d97634866b99714068e8f5c24/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:36.067400 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4126de-a1d7-463f-a923-3bcff9f703e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=2bf135d016d2468ea74f657e5ba96ce0, user_id=8b7d729d97634866b99714068e8f5c24, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:36.068838 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4126de-a1d7-463f-a923-3bcff9f703e8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:36.083493 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 562/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:30:35 2022] PUT /identity/v3/projects/2bf135d016d2468ea74f657e5ba96ce0/users/8b7d729d97634866b99714068e8f5c24/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:30:38.176946 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721b760-2118-4ed7-ad95-96bae4fa7729 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:38.177474 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721b760-2118-4ed7-ad95-96bae4fa7729 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:38.177474 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721b760-2118-4ed7-ad95-96bae4fa7729 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:38.178901 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 592/1154] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:38.184704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b45aad-c965-4ba9-b045-1fff5a7f7101 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:38.185108 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b45aad-c965-4ba9-b045-1fff5a7f7101 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:38.185255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b45aad-c965-4ba9-b045-1fff5a7f7101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:38.202472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-09b45aad-c965-4ba9-b045-1fff5a7f7101 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:38.203660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-09b45aad-c965-4ba9-b045-1fff5a7f7101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:38.268101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 563/1155] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:38.292013 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721b760-2118-4ed7-ad95-96bae4fa7729 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:38.397763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721b760-2118-4ed7-ad95-96bae4fa7729 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:38.420883 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:38.433063 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef75c4-6ce9-40aa-8cbc-122a1a7531cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:38.433784 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef75c4-6ce9-40aa-8cbc-122a1a7531cd None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:38.434097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef75c4-6ce9-40aa-8cbc-122a1a7531cd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:38.434370 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef75c4-6ce9-40aa-8cbc-122a1a7531cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:38.444145 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef75c4-6ce9-40aa-8cbc-122a1a7531cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:30:38.451850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-e3ef75c4-6ce9-40aa-8cbc-122a1a7531cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:38.451850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:38.451850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:38.451850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:38.451850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:38.451850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:38.451850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:38.451850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:38.451850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:38.451850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:38.451850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:38.451850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:38.451850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:30:38.451850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:30:38.451850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:30:38.451850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:30:38.452994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:30:38.452994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:30:38.452994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:30:38.452994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:30:38.452994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:38.452994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:38.452994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:38.452994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:38.452994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:38.452994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:38.452994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:38.452994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:38.452994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:38.452994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:38.452994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:38.452994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:38.454164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:38.454164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:38.454164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:38.454164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:38.454164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:30:38.454164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:30:38.454164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:30:38.454164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:30:38.454164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:30:38.454164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:30:38.454164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:38.454164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:30:38.454164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 593/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:30:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:38.459065 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b45aad-c965-4ba9-b045-1fff5a7f7101 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:38.498607 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:38.557637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b45aad-c965-4ba9-b045-1fff5a7f7101 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:38.589741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681b20c-5995-44d6-9b2e-6b8a4871ac69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:38.590431 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681b20c-5995-44d6-9b2e-6b8a4871ac69 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:38.590713 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681b20c-5995-44d6-9b2e-6b8a4871ac69 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:38.590986 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681b20c-5995-44d6-9b2e-6b8a4871ac69 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:38.591722 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681b20c-5995-44d6-9b2e-6b8a4871ac69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:38.592019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681b20c-5995-44d6-9b2e-6b8a4871ac69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:38.593375 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681b20c-5995-44d6-9b2e-6b8a4871ac69 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:38.602337 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 564/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:30:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:38.608312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef75c4-6ce9-40aa-8cbc-122a1a7531cd None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:38.710163 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef75c4-6ce9-40aa-8cbc-122a1a7531cd None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:38.757230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4a274-1b79-4efb-9468-ddc8460d1470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:38.758642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4a274-1b79-4efb-9468-ddc8460d1470 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:38.758940 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4a274-1b79-4efb-9468-ddc8460d1470 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:38.759270 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4a274-1b79-4efb-9468-ddc8460d1470 None admin] PATH_INFO: `/v3/users/8b7d729d97634866b99714068e8f5c24` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:38.760822 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad4a274-1b79-4efb-9468-ddc8460d1470 None admin] RBAC: Authorizing `identity:get_user(user_id=8b7d729d97634866b99714068e8f5c24)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:38.763132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad4a274-1b79-4efb-9468-ddc8460d1470 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:38.765420 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 594/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:30:38 2022] GET /identity/v3/users/8b7d729d97634866b99714068e8f5c24 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:38.771169 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681b20c-5995-44d6-9b2e-6b8a4871ac69 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:38.866155 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:38.869199 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681b20c-5995-44d6-9b2e-6b8a4871ac69 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:38.902170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d3f5e-4d00-4e67-b43a-b531f1c9837c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:38.902810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d3f5e-4d00-4e67-b43a-b531f1c9837c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:38.903091 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d3f5e-4d00-4e67-b43a-b531f1c9837c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:38.903365 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d3f5e-4d00-4e67-b43a-b531f1c9837c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:38.968164 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d3f5e-4d00-4e67-b43a-b531f1c9837c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:38.969532 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d3f5e-4d00-4e67-b43a-b531f1c9837c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:39.072378 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:39.076382 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 565/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:30:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:30:39.082320 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4a274-1b79-4efb-9468-ddc8460d1470 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:39.130371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:39.178973 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4a274-1b79-4efb-9468-ddc8460d1470 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:39.211855 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4bf5-3850-456c-93cc-a3f064afc37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:39.212714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4bf5-3850-456c-93cc-a3f064afc37d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:39.212979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4bf5-3850-456c-93cc-a3f064afc37d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:39.213251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4bf5-3850-456c-93cc-a3f064afc37d None admin] PATH_INFO: `/v3/projects/2bf135d016d2468ea74f657e5ba96ce0` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:39.214433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e4bf5-3850-456c-93cc-a3f064afc37d None admin] RBAC: Authorizing `identity:get_project(project_id=2bf135d016d2468ea74f657e5ba96ce0)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:39.215858 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e4bf5-3850-456c-93cc-a3f064afc37d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:39.217528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 595/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:30:39 2022] GET /identity/v3/projects/2bf135d016d2468ea74f657e5ba96ce0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:39.223724 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d3f5e-4d00-4e67-b43a-b531f1c9837c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:39.319457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d3f5e-4d00-4e67-b43a-b531f1c9837c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:39.352911 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43557123-a5bc-4081-9d6f-faca89798368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:39.353556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43557123-a5bc-4081-9d6f-faca89798368 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:39.353840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43557123-a5bc-4081-9d6f-faca89798368 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:39.354151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43557123-a5bc-4081-9d6f-faca89798368 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:39.354974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43557123-a5bc-4081-9d6f-faca89798368 None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=2bf135d016d2468ea74f657e5ba96ce0, user.id=8b7d729d97634866b99714068e8f5c24) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:39.355289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43557123-a5bc-4081-9d6f-faca89798368 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:39.356728 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43557123-a5bc-4081-9d6f-faca89798368 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:39.362436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 566/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:30:39 2022] GET /identity/v3/role_assignments?user.id=8b7d729d97634866b99714068e8f5c24&scope.project.id=2bf135d016d2468ea74f657e5ba96ce0&role.id=864a6966630544e8bd68bcafba905c6f => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:41.386667 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751fb9a-1b77-448f-9d3b-42bafeca15b0 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:41.387194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751fb9a-1b77-448f-9d3b-42bafeca15b0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:41.387194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751fb9a-1b77-448f-9d3b-42bafeca15b0 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:41.388561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 596/1162] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:41.393583 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da736dc5-4bc1-4f3c-ad81-5e9f992fb64f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:41.394015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da736dc5-4bc1-4f3c-ad81-5e9f992fb64f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:41.394167 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da736dc5-4bc1-4f3c-ad81-5e9f992fb64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:41.410411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-da736dc5-4bc1-4f3c-ad81-5e9f992fb64f None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:41.411597 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-da736dc5-4bc1-4f3c-ad81-5e9f992fb64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:41.468122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:41.477103 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 567/1163] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:41.499876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751fb9a-1b77-448f-9d3b-42bafeca15b0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:41.597832 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751fb9a-1b77-448f-9d3b-42bafeca15b0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:41.615314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:41.633020 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1460b-4dfd-4834-85e6-c041ef493ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:41.633669 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1460b-4dfd-4834-85e6-c041ef493ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:41.633979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1460b-4dfd-4834-85e6-c041ef493ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:41.634270 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1460b-4dfd-4834-85e6-c041ef493ac5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:41.702728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1460b-4dfd-4834-85e6-c041ef493ac5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:41.704151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1460b-4dfd-4834-85e6-c041ef493ac5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:41.800281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 597/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:30:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:30:41.806331 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da736dc5-4bc1-4f3c-ad81-5e9f992fb64f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:41.900990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da736dc5-4bc1-4f3c-ad81-5e9f992fb64f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:41.934029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fcc9a8-be18-411d-a98f-267a9622d4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:41.934835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fcc9a8-be18-411d-a98f-267a9622d4ec None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:41.935121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fcc9a8-be18-411d-a98f-267a9622d4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:41.935399 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fcc9a8-be18-411d-a98f-267a9622d4ec None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:41.941930 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fcc9a8-be18-411d-a98f-267a9622d4ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:41.943392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fcc9a8-be18-411d-a98f-267a9622d4ec None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:41.957348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-54fcc9a8-be18-411d-a98f-267a9622d4ec None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 12 13:30:41.957348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:41.957348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 12 13:30:41.957348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 12 13:30:41.957348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:30:41.957348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:30:41.957348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:30:41.957348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:30:41.957348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 12 13:30:41.957348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:41.957348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:30:41.957348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:41.957348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:41.957348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:41.957348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:41.957348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:41.957348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:41.958599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:41.958599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:41.958599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:41.958599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:41.958599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:41.958599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:41.958599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:30:41.958599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:30:41.958599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:30:41.958599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:30:41.958599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:41.958599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:41.958599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:41.958599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:41.958599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:41.958599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:41.959635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:41.959635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:41.959635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:41.959635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:41.959635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:41.959635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:41.959635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:41.959635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:41.959635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:41.959635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:41.959635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 12 13:30:41.959635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:30:41.959635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 12 13:30:41.959635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:41.959635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 568/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jul 12 13:30:41 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:41.964949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1460b-4dfd-4834-85e6-c041ef493ac5 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:41.983265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:42.062719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1460b-4dfd-4834-85e6-c041ef493ac5 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:42.096874 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7bea2-7405-4a91-b032-84ed83d6d78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:42.097528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7bea2-7405-4a91-b032-84ed83d6d78f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:42.097814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7bea2-7405-4a91-b032-84ed83d6d78f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:42.098117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7bea2-7405-4a91-b032-84ed83d6d78f None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:42.098829 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7bea2-7405-4a91-b032-84ed83d6d78f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:42.099134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7bea2-7405-4a91-b032-84ed83d6d78f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:42.100507 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7bea2-7405-4a91-b032-84ed83d6d78f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:42.107796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 598/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Jul 12 13:30:41 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:44.130317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c0438-ad55-4ad6-b717-630c3013780d None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:44.130841 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c0438-ad55-4ad6-b717-630c3013780d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:44.130841 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c0438-ad55-4ad6-b717-630c3013780d None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:44.132241 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 569/1167] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:44.137492 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bf2ca-f474-484f-8c5b-fab482cf287c None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:44.137896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bf2ca-f474-484f-8c5b-fab482cf287c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:44.138074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bf2ca-f474-484f-8c5b-fab482cf287c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:44.154852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-319bf2ca-f474-484f-8c5b-fab482cf287c None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:44.156035 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-319bf2ca-f474-484f-8c5b-fab482cf287c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:44.220850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 599/1168] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:44.243964 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c0438-ad55-4ad6-b717-630c3013780d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:44.300970 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:44.348172 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c0438-ad55-4ad6-b717-630c3013780d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:44.382509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c7835-61a9-4803-982f-7373846aafd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:44.383618 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c7835-61a9-4803-982f-7373846aafd5 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:44.383906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c7835-61a9-4803-982f-7373846aafd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:44.384181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c7835-61a9-4803-982f-7373846aafd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:44.384926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c7835-61a9-4803-982f-7373846aafd5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:44.386330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c7835-61a9-4803-982f-7373846aafd5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:44.401866 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 570/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Tue Jul 12 13:30:44 2022] POST /identity/v3/domains => generated 267 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:30:46.435149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efddd9-b0fe-4bed-84a5-b8f157e2a1c0 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:46.435789 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efddd9-b0fe-4bed-84a5-b8f157e2a1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:46.435789 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efddd9-b0fe-4bed-84a5-b8f157e2a1c0 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:46.437085 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 600/1170] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:46.442622 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a5449-23da-4d11-9102-58bf80c1add1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:46.443080 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a5449-23da-4d11-9102-58bf80c1add1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:46.443181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a5449-23da-4d11-9102-58bf80c1add1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:46.459587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-be4a5449-23da-4d11-9102-58bf80c1add1 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:46.461262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-be4a5449-23da-4d11-9102-58bf80c1add1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:46.527804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 571/1171] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:46.550539 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efddd9-b0fe-4bed-84a5-b8f157e2a1c0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:46.565366 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:46.652497 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efddd9-b0fe-4bed-84a5-b8f157e2a1c0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:46.686345 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2dc6e3-f4eb-4972-a42b-6214c4b127b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:46.687120 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dc6e3-f4eb-4972-a42b-6214c4b127b9 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:46.687397 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dc6e3-f4eb-4972-a42b-6214c4b127b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:46.687671 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dc6e3-f4eb-4972-a42b-6214c4b127b9 None admin] PATH_INFO: `/v3/domains/602cc8c5e7584eea87b4b07011405227` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:46.698525 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2dc6e3-f4eb-4972-a42b-6214c4b127b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=602cc8c5e7584eea87b4b07011405227)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:46.699978 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2dc6e3-f4eb-4972-a42b-6214c4b127b9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:46.701601 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 601/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Tue Jul 12 13:30:46 2022] GET /identity/v3/domains/602cc8c5e7584eea87b4b07011405227 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:46.708719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a5449-23da-4d11-9102-58bf80c1add1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:46.773906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:46.832240 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a5449-23da-4d11-9102-58bf80c1add1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:46.866954 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0f036-4785-45f5-88bb-83c1e8873a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:46.867631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0f036-4785-45f5-88bb-83c1e8873a4b None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:46.867917 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0f036-4785-45f5-88bb-83c1e8873a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:46.868194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0f036-4785-45f5-88bb-83c1e8873a4b None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:46.868901 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0f036-4785-45f5-88bb-83c1e8873a4b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:46.870442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0f036-4785-45f5-88bb-83c1e8873a4b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:46.901804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 572/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jul 12 13:30:46 2022] POST /identity/v3/projects => generated 361 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:30:48.952980 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5b175-8792-428e-9fc2-d074715743c5 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:48.953503 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5b175-8792-428e-9fc2-d074715743c5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:48.953632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5b175-8792-428e-9fc2-d074715743c5 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:48.954945 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 602/1174] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:48.960093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfa510-19bd-4992-9e0d-7794ebc73a33 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:48.960525 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfa510-19bd-4992-9e0d-7794ebc73a33 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:48.960647 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfa510-19bd-4992-9e0d-7794ebc73a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:48.977299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f0dfa510-19bd-4992-9e0d-7794ebc73a33 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:48.978528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f0dfa510-19bd-4992-9e0d-7794ebc73a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:49.047040 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 573/1175] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:49.072189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da5b175-8792-428e-9fc2-d074715743c5 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:49.142081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:49.172848 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da5b175-8792-428e-9fc2-d074715743c5 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:49.205020 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb78a9-ea80-4316-a5b7-d581b3980355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:49.205869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb78a9-ea80-4316-a5b7-d581b3980355 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:49.206184 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb78a9-ea80-4316-a5b7-d581b3980355 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:49.206464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb78a9-ea80-4316-a5b7-d581b3980355 None admin] PATH_INFO: `/v3/domains/602cc8c5e7584eea87b4b07011405227` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:49.207639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb78a9-ea80-4316-a5b7-d581b3980355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=602cc8c5e7584eea87b4b07011405227)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:49.209075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb78a9-ea80-4316-a5b7-d581b3980355 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:49.211237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 603/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Tue Jul 12 13:30:49 2022] GET /identity/v3/domains/602cc8c5e7584eea87b4b07011405227 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:49.216955 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfa510-19bd-4992-9e0d-7794ebc73a33 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:49.243162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:49.313221 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfa510-19bd-4992-9e0d-7794ebc73a33 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:49.347699 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bccbee4-d607-4ea6-9625-e047d124f738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:49.348352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccbee4-d607-4ea6-9625-e047d124f738 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:49.348627 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccbee4-d607-4ea6-9625-e047d124f738 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:49.348898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccbee4-d607-4ea6-9625-e047d124f738 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:49.349609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bccbee4-d607-4ea6-9625-e047d124f738 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:49.351129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bccbee4-d607-4ea6-9625-e047d124f738 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:49.371604 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 574/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:30:49 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:30:51.411596 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48472bfa-9ce7-4752-a609-9b2c02c783ff None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:51.412146 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48472bfa-9ce7-4752-a609-9b2c02c783ff None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:51.412146 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48472bfa-9ce7-4752-a609-9b2c02c783ff None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:51.413513 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 604/1178] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:51.418640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc39cd-8029-44e2-ac7c-a058efce356d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:51.419080 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc39cd-8029-44e2-ac7c-a058efce356d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:51.419201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc39cd-8029-44e2-ac7c-a058efce356d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:51.435994 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e9dc39cd-8029-44e2-ac7c-a058efce356d None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:51.437182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e9dc39cd-8029-44e2-ac7c-a058efce356d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:51.502130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 575/1179] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:51.525792 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48472bfa-9ce7-4752-a609-9b2c02c783ff None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:51.626441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48472bfa-9ce7-4752-a609-9b2c02c783ff None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:51.659743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28e7c1-30b8-4845-a1f6-70694f5424a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:51.660523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28e7c1-30b8-4845-a1f6-70694f5424a0 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:51.660812 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28e7c1-30b8-4845-a1f6-70694f5424a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:51.661087 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28e7c1-30b8-4845-a1f6-70694f5424a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:51.669683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:51.673388 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28e7c1-30b8-4845-a1f6-70694f5424a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:30:51.681926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-cd28e7c1-30b8-4845-a1f6-70694f5424a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:51.681926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:51.681926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:51.681926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:51.681926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:51.681926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:51.681926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:51.681926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:51.681926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:51.681926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:51.681926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:51.681926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:51.681926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:30:51.681926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:30:51.681926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:30:51.681926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:30:51.683190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:30:51.683190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:30:51.683190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:30:51.683190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:30:51.683190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:51.683190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:51.683190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:51.683190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:51.683190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:51.683190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:51.683190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:51.683190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:51.683190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:51.683190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:51.683190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:51.683190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:51.684162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:51.684162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:51.684162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:51.684162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:51.684162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:30:51.684162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:30:51.684162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:30:51.684162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:30:51.684162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:30:51.684162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:30:51.684162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:51.684162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:30:51.684162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 605/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:30:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:51.690678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc39cd-8029-44e2-ac7c-a058efce356d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:51.705405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:51.787975 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc39cd-8029-44e2-ac7c-a058efce356d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:51.822124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44302ded-06de-4f66-b6c7-054ce789c366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:51.822778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44302ded-06de-4f66-b6c7-054ce789c366 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:51.823061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44302ded-06de-4f66-b6c7-054ce789c366 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:51.823336 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44302ded-06de-4f66-b6c7-054ce789c366 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:51.824080 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44302ded-06de-4f66-b6c7-054ce789c366 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:51.824382 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44302ded-06de-4f66-b6c7-054ce789c366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:51.825741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44302ded-06de-4f66-b6c7-054ce789c366 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:51.833541 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 576/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:30:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:51.839319 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28e7c1-30b8-4845-a1f6-70694f5424a0 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:51.938816 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28e7c1-30b8-4845-a1f6-70694f5424a0 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:51.971107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbf1ea-70f7-452d-b634-ea7c39c94f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:51.971970 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbf1ea-70f7-452d-b634-ea7c39c94f40 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:51.972255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbf1ea-70f7-452d-b634-ea7c39c94f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:51.972528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbf1ea-70f7-452d-b634-ea7c39c94f40 None admin] PATH_INFO: `/v3/users/ee79070bb7ea4753887255d97afbfbe7` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:51.985831 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbf1ea-70f7-452d-b634-ea7c39c94f40 None admin] RBAC: Authorizing `identity:get_user(user_id=ee79070bb7ea4753887255d97afbfbe7)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:51.987318 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbf1ea-70f7-452d-b634-ea7c39c94f40 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:51.988985 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 606/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:30:51 2022] GET /identity/v3/users/ee79070bb7ea4753887255d97afbfbe7 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:51.995422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44302ded-06de-4f66-b6c7-054ce789c366 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:52.073704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:52.105519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44302ded-06de-4f66-b6c7-054ce789c366 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:52.140591 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dffe2-ddc0-4598-863f-61b74867b209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:52.141267 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dffe2-ddc0-4598-863f-61b74867b209 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:52.141549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dffe2-ddc0-4598-863f-61b74867b209 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:52.141820 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dffe2-ddc0-4598-863f-61b74867b209 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:52.220718 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483dffe2-ddc0-4598-863f-61b74867b209 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:52.222126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483dffe2-ddc0-4598-863f-61b74867b209 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:52.297524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:52.323691 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 577/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:30:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:30:52.329907 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbf1ea-70f7-452d-b634-ea7c39c94f40 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:52.403067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:52.449652 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbf1ea-70f7-452d-b634-ea7c39c94f40 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:52.490806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e9ee4-4b8d-461d-b8be-41883529e49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:52.491575 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e9ee4-4b8d-461d-b8be-41883529e49e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:52.491891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e9ee4-4b8d-461d-b8be-41883529e49e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:52.492158 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e9ee4-4b8d-461d-b8be-41883529e49e None admin] PATH_INFO: `/v3/projects/7d5899644f41493086e2a754516db440` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:52.493360 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e9ee4-4b8d-461d-b8be-41883529e49e None admin] RBAC: Authorizing `identity:get_project(project_id=7d5899644f41493086e2a754516db440)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:52.494828 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e9ee4-4b8d-461d-b8be-41883529e49e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:52.496538 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 607/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:30:52 2022] GET /identity/v3/projects/7d5899644f41493086e2a754516db440 => generated 361 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:52.502941 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dffe2-ddc0-4598-863f-61b74867b209 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:52.620639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dffe2-ddc0-4598-863f-61b74867b209 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:52.658484 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2f0f2-8d55-45ab-a834-e3d6e005698e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:52.659141 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2f0f2-8d55-45ab-a834-e3d6e005698e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:52.659425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2f0f2-8d55-45ab-a834-e3d6e005698e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:52.659702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2f0f2-8d55-45ab-a834-e3d6e005698e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:52.660529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2f0f2-8d55-45ab-a834-e3d6e005698e None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=7d5899644f41493086e2a754516db440, user.id=ee79070bb7ea4753887255d97afbfbe7) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:52.660842 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2f0f2-8d55-45ab-a834-e3d6e005698e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:52.662307 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2f0f2-8d55-45ab-a834-e3d6e005698e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:52.668140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 578/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:30:52 2022] GET /identity/v3/role_assignments?user.id=ee79070bb7ea4753887255d97afbfbe7&scope.project.id=7d5899644f41493086e2a754516db440&role.id=864a6966630544e8bd68bcafba905c6f => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:54.697368 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b71bd6-0f7a-4a74-bedd-e7e79967b747 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:54.697904 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b71bd6-0f7a-4a74-bedd-e7e79967b747 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:54.697904 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b71bd6-0f7a-4a74-bedd-e7e79967b747 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:54.699312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 608/1186] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:54.704660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d50b9-b6c5-4a35-8471-d3b9aa12b3f8 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:54.705072 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d50b9-b6c5-4a35-8471-d3b9aa12b3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:54.705185 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d50b9-b6c5-4a35-8471-d3b9aa12b3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:54.722253 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0a4d50b9-b6c5-4a35-8471-d3b9aa12b3f8 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:54.723671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0a4d50b9-b6c5-4a35-8471-d3b9aa12b3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:54.754090 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:54.794315 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 579/1187] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:54.817277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b71bd6-0f7a-4a74-bedd-e7e79967b747 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:54.920906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b71bd6-0f7a-4a74-bedd-e7e79967b747 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:54.947217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:54.956686 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff30932-a07c-4f89-9f39-fc349814b10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:54.957407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff30932-a07c-4f89-9f39-fc349814b10e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:54.957695 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff30932-a07c-4f89-9f39-fc349814b10e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:54.957980 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff30932-a07c-4f89-9f39-fc349814b10e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:54.968747 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff30932-a07c-4f89-9f39-fc349814b10e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:30:54.977201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-4ff30932-a07c-4f89-9f39-fc349814b10e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:54.977201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:54.977201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:54.977201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:54.977201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:54.977201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:54.977201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:54.977201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:54.977201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:54.977201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:54.977201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:54.977201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:54.977201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:30:54.977201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:30:54.977201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:30:54.977201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:30:54.978407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:30:54.978407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:30:54.978407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:30:54.978407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:30:54.978407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:54.978407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:54.978407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:54.978407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:54.978407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:54.978407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:54.978407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:54.978407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:54.978407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:54.978407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:54.978407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:54.978407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:54.979441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:54.979441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:54.979441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:54.979441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:54.979441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:30:54.979441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:30:54.979441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:30:54.979441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:30:54.979441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:30:54.979441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:30:54.979441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:54.979441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:30:54.979441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 609/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:30:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:54.984647 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d50b9-b6c5-4a35-8471-d3b9aa12b3f8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:55.085988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d50b9-b6c5-4a35-8471-d3b9aa12b3f8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:55.122189 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428da1f-522a-4562-9e10-d6f320a91177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:55.122869 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428da1f-522a-4562-9e10-d6f320a91177 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:55.123156 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428da1f-522a-4562-9e10-d6f320a91177 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:55.123433 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428da1f-522a-4562-9e10-d6f320a91177 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:55.124187 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d428da1f-522a-4562-9e10-d6f320a91177 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:55.124531 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d428da1f-522a-4562-9e10-d6f320a91177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:55.126198 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d428da1f-522a-4562-9e10-d6f320a91177 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:55.135142 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 580/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:30:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:55.141156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff30932-a07c-4f89-9f39-fc349814b10e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:55.245705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff30932-a07c-4f89-9f39-fc349814b10e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:55.281254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051c398-9402-486c-8830-2b845144759a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:55.282154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051c398-9402-486c-8830-2b845144759a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:55.282440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051c398-9402-486c-8830-2b845144759a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:55.282717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051c398-9402-486c-8830-2b845144759a None admin] PATH_INFO: `/v3/users/ee79070bb7ea4753887255d97afbfbe7` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:55.283917 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051c398-9402-486c-8830-2b845144759a None admin] RBAC: Authorizing `identity:get_user(user_id=ee79070bb7ea4753887255d97afbfbe7)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:55.285366 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051c398-9402-486c-8830-2b845144759a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:55.287051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 610/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:30:55 2022] GET /identity/v3/users/ee79070bb7ea4753887255d97afbfbe7 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:55.293685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428da1f-522a-4562-9e10-d6f320a91177 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:55.325668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:55.395442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428da1f-522a-4562-9e10-d6f320a91177 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:55.433232 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b9f9b-22c1-4072-94bd-dfe842767e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:55.434139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b9f9b-22c1-4072-94bd-dfe842767e5c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:55.434427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b9f9b-22c1-4072-94bd-dfe842767e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:55.434704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b9f9b-22c1-4072-94bd-dfe842767e5c None admin] PATH_INFO: `/v3/projects/7d5899644f41493086e2a754516db440` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:55.435895 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b9f9b-22c1-4072-94bd-dfe842767e5c None admin] RBAC: Authorizing `identity:get_project(project_id=7d5899644f41493086e2a754516db440)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:55.437370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b9f9b-22c1-4072-94bd-dfe842767e5c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:55.439141 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 581/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:30:55 2022] GET /identity/v3/projects/7d5899644f41493086e2a754516db440 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:55.445061 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051c398-9402-486c-8830-2b845144759a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:55.501612 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:55.549521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051c398-9402-486c-8830-2b845144759a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:55.584040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a206299-a149-4e94-b47f-7b4ccb1bd8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:55.584932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a206299-a149-4e94-b47f-7b4ccb1bd8aa None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:55.585217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a206299-a149-4e94-b47f-7b4ccb1bd8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:55.586067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a206299-a149-4e94-b47f-7b4ccb1bd8aa None admin] PATH_INFO: `/v3/projects/7d5899644f41493086e2a754516db440/users/ee79070bb7ea4753887255d97afbfbe7/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:55.588048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a206299-a149-4e94-b47f-7b4ccb1bd8aa None admin] RBAC: Authorizing `identity:create_grant(project_id=7d5899644f41493086e2a754516db440, user_id=ee79070bb7ea4753887255d97afbfbe7, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:55.589496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a206299-a149-4e94-b47f-7b4ccb1bd8aa None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:55.603647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 611/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:30:55 2022] PUT /identity/v3/projects/7d5899644f41493086e2a754516db440/users/ee79070bb7ea4753887255d97afbfbe7/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:30:57.643687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb5811-9315-48d1-be8d-0c9d8f8d5741 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:57.644238 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb5811-9315-48d1-be8d-0c9d8f8d5741 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:57.644238 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb5811-9315-48d1-be8d-0c9d8f8d5741 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:57.645596 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 582/1193] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:30:57.651010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69988b-0c75-440a-906f-d512af0a6e54 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:57.651538 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69988b-0c75-440a-906f-d512af0a6e54 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:57.651686 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69988b-0c75-440a-906f-d512af0a6e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:57.671388 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-cf69988b-0c75-440a-906f-d512af0a6e54 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:57.672611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cf69988b-0c75-440a-906f-d512af0a6e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:57.743276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 612/1194] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:30:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:30:57.770363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb5811-9315-48d1-be8d-0c9d8f8d5741 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:57.881725 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:57.886995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb5811-9315-48d1-be8d-0c9d8f8d5741 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:57.927738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151580e-a647-45e4-a24f-f17b3a108505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:57.928428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151580e-a647-45e4-a24f-f17b3a108505 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:57.928710 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151580e-a647-45e4-a24f-f17b3a108505 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:57.928988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151580e-a647-45e4-a24f-f17b3a108505 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:57.941090 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151580e-a647-45e4-a24f-f17b3a108505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:30:57.948926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-9151580e-a647-45e4-a24f-f17b3a108505 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:57.948926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:30:57.948926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:30:57.948926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:30:57.948926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:30:57.948926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:30:57.948926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:30:57.948926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:30:57.948926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:30:57.948926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:30:57.948926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:30:57.948926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:30:57.948926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:30:57.948926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:30:57.948926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:30:57.948926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:30:57.950063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:30:57.950063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:30:57.950063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:30:57.950063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:30:57.950063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:30:57.950063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:30:57.950063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:30:57.950063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:30:57.950063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:30:57.950063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:30:57.950063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:30:57.950063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:30:57.950063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:30:57.950063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:30:57.950063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:30:57.950063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:30:57.950948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:30:57.950948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:30:57.950948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:30:57.950948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:30:57.950948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:30:57.950948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:30:57.950948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:30:57.950948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:30:57.950948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:30:57.950948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:30:57.950948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:30:57.950948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:30:57.950948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 583/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:30:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:30:57.956622 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69988b-0c75-440a-906f-d512af0a6e54 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:58.012317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:58.070742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69988b-0c75-440a-906f-d512af0a6e54 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:58.111659 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd345640-2d9b-47be-b23e-57732eaf2aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:58.112349 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd345640-2d9b-47be-b23e-57732eaf2aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:58.112634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd345640-2d9b-47be-b23e-57732eaf2aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:58.112910 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd345640-2d9b-47be-b23e-57732eaf2aa6 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:58.113661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd345640-2d9b-47be-b23e-57732eaf2aa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:58.113989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd345640-2d9b-47be-b23e-57732eaf2aa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:58.115490 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd345640-2d9b-47be-b23e-57732eaf2aa6 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:58.125257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 613/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:30:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:30:58.131248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151580e-a647-45e4-a24f-f17b3a108505 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:58.240571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151580e-a647-45e4-a24f-f17b3a108505 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:58.276983 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e43e9-f740-4431-8447-9b292461cc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:58.277859 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e43e9-f740-4431-8447-9b292461cc81 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:58.278176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e43e9-f740-4431-8447-9b292461cc81 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:58.278455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e43e9-f740-4431-8447-9b292461cc81 None admin] PATH_INFO: `/v3/users/ee79070bb7ea4753887255d97afbfbe7` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:58.279932 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e43e9-f740-4431-8447-9b292461cc81 None admin] RBAC: Authorizing `identity:get_user(user_id=ee79070bb7ea4753887255d97afbfbe7)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:58.281126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e43e9-f740-4431-8447-9b292461cc81 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:58.283078 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 584/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jul 12 13:30:58 2022] GET /identity/v3/users/ee79070bb7ea4753887255d97afbfbe7 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:58.289223 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd345640-2d9b-47be-b23e-57732eaf2aa6 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:58.393140 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd345640-2d9b-47be-b23e-57732eaf2aa6 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:58.400818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:58.429395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c8243-c28b-42ff-b55f-8b3f92d374eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:58.430070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c8243-c28b-42ff-b55f-8b3f92d374eb None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:58.430356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c8243-c28b-42ff-b55f-8b3f92d374eb None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:58.430633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c8243-c28b-42ff-b55f-8b3f92d374eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:58.513860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c8243-c28b-42ff-b55f-8b3f92d374eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:58.515862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c8243-c28b-42ff-b55f-8b3f92d374eb None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:58.621552 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 614/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:30:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:30:58.627707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e43e9-f740-4431-8447-9b292461cc81 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:58.670781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:58.759329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e43e9-f740-4431-8447-9b292461cc81 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:58.798286 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19444e-984a-419f-8b6f-9e12083e3fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:58.799071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19444e-984a-419f-8b6f-9e12083e3fbe None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:58.799347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19444e-984a-419f-8b6f-9e12083e3fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:58.799615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19444e-984a-419f-8b6f-9e12083e3fbe None admin] PATH_INFO: `/v3/projects/7d5899644f41493086e2a754516db440` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:58.800812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd19444e-984a-419f-8b6f-9e12083e3fbe None admin] RBAC: Authorizing `identity:get_project(project_id=7d5899644f41493086e2a754516db440)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:58.802284 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd19444e-984a-419f-8b6f-9e12083e3fbe None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:58.804234 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 585/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jul 12 13:30:58 2022] GET /identity/v3/projects/7d5899644f41493086e2a754516db440 => generated 361 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:30:58.813246 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c8243-c28b-42ff-b55f-8b3f92d374eb None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:30:58.822813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:58.915510 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c8243-c28b-42ff-b55f-8b3f92d374eb None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:30:58.953367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36576e-5bc5-4f46-aed1-1aa2a8b94b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:30:58.954052 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36576e-5bc5-4f46-aed1-1aa2a8b94b8b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:58.954341 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36576e-5bc5-4f46-aed1-1aa2a8b94b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:58.954620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36576e-5bc5-4f46-aed1-1aa2a8b94b8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:58.955450 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36576e-5bc5-4f46-aed1-1aa2a8b94b8b None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=7d5899644f41493086e2a754516db440, user.id=ee79070bb7ea4753887255d97afbfbe7) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:30:58.955766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36576e-5bc5-4f46-aed1-1aa2a8b94b8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:30:58.957213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36576e-5bc5-4f46-aed1-1aa2a8b94b8b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:30:58.963335 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 615/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:30:58 2022] GET /identity/v3/role_assignments?user.id=ee79070bb7ea4753887255d97afbfbe7&scope.project.id=7d5899644f41493086e2a754516db440&role.id=864a6966630544e8bd68bcafba905c6f => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:31:08.789667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=53 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:08.960744 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=65 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:54.569794 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb148fa-11a8-4a87-a3de-c4adcc08d7f4 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:54.569794 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb148fa-11a8-4a87-a3de-c4adcc08d7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:54.569794 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb148fa-11a8-4a87-a3de-c4adcc08d7f4 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:54.569794 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 586/1201] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:31:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:31:54.576415 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7485449-6a15-4cce-b150-2e30bf7377b1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:54.576887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7485449-6a15-4cce-b150-2e30bf7377b1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:54.576981 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7485449-6a15-4cce-b150-2e30bf7377b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:54.601985 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-e7485449-6a15-4cce-b150-2e30bf7377b1 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:31:54.603505 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e7485449-6a15-4cce-b150-2e30bf7377b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:54.700228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 616/1202] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:31:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:31:54.733759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb148fa-11a8-4a87-a3de-c4adcc08d7f4 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:54.870308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb148fa-11a8-4a87-a3de-c4adcc08d7f4 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:54.915907 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188da43-ba4a-40a4-8e79-3323d6978f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:54.917127 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188da43-ba4a-40a4-8e79-3323d6978f86 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:54.917427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188da43-ba4a-40a4-8e79-3323d6978f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:54.917700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188da43-ba4a-40a4-8e79-3323d6978f86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:54.934137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188da43-ba4a-40a4-8e79-3323d6978f86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:54.936016 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188da43-ba4a-40a4-8e79-3323d6978f86 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:54.958988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-3188da43-ba4a-40a4-8e79-3323d6978f86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:31:54.958988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:31:54.958988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:31:54.958988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:31:54.958988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:31:54.958988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:31:54.958988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:31:54.958988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:31:54.958988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:31:54.958988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:31:54.958988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:31:54.958988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:31:54.958988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:31:54.958988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:31:54.958988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:31:54.960074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:31:54.960074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:31:54.960074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:31:54.960074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:31:54.960074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:31:54.960074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:31:54.960074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:31:54.960074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:31:54.960074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:31:54.960074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:31:54.960074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:31:54.960074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:31:54.960074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:31:54.960074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:31:54.960074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:31:54.960074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:31:54.960074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:31:54.961061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:31:54.961061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:31:54.961061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:31:54.961061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:31:54.961061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 587/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:31:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:31:54.966683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7485449-6a15-4cce-b150-2e30bf7377b1 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:55.096178 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7485449-6a15-4cce-b150-2e30bf7377b1 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:55.140130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:55.148812 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b25c8-7634-4d5b-8fec-84ef841152cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:55.149486 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b25c8-7634-4d5b-8fec-84ef841152cb None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:55.149761 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b25c8-7634-4d5b-8fec-84ef841152cb None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:55.150069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b25c8-7634-4d5b-8fec-84ef841152cb None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:55.150809 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b25c8-7634-4d5b-8fec-84ef841152cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:31:55.151096 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b25c8-7634-4d5b-8fec-84ef841152cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:55.152488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b25c8-7634-4d5b-8fec-84ef841152cb None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:55.165988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 617/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:31:55.171782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188da43-ba4a-40a4-8e79-3323d6978f86 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:55.239373 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:55.303248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188da43-ba4a-40a4-8e79-3323d6978f86 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:55.345724 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486d166-a38e-4213-b912-326e66fba9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:55.346664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486d166-a38e-4213-b912-326e66fba9ba None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:55.346795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486d166-a38e-4213-b912-326e66fba9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:55.346977 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486d166-a38e-4213-b912-326e66fba9ba None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:55.347711 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486d166-a38e-4213-b912-326e66fba9ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:55.350060 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486d166-a38e-4213-b912-326e66fba9ba None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:55.410005 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 588/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jul 12 13:31:55 2022] POST /identity/v3/users => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:31:57.769253 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5407e-06e2-465f-91d5-1eb0f0b85fe1 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:57.769823 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5407e-06e2-465f-91d5-1eb0f0b85fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:57.769823 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5407e-06e2-465f-91d5-1eb0f0b85fe1 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:57.771651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 618/1206] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:31:57.777663 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ade28-8903-4833-905a-1118376ef33d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:57.778456 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ade28-8903-4833-905a-1118376ef33d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:57.778456 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ade28-8903-4833-905a-1118376ef33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:57.799732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ec0ade28-8903-4833-905a-1118376ef33d None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:31:57.801231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ec0ade28-8903-4833-905a-1118376ef33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:57.891865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 589/1207] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:31:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:31:57.916730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5407e-06e2-465f-91d5-1eb0f0b85fe1 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:58.047067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5407e-06e2-465f-91d5-1eb0f0b85fe1 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:58.091685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ff51f-e4bf-4abc-b8c1-7f3ac4ebd4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:58.092606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ff51f-e4bf-4abc-b8c1-7f3ac4ebd4ca None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:58.092938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ff51f-e4bf-4abc-b8c1-7f3ac4ebd4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:58.093210 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ff51f-e4bf-4abc-b8c1-7f3ac4ebd4ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:58.106046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ff51f-e4bf-4abc-b8c1-7f3ac4ebd4ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:31:58.115257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-c19ff51f-e4bf-4abc-b8c1-7f3ac4ebd4ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:31:58.115257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:31:58.115257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:31:58.115257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:31:58.115257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:31:58.115257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:31:58.115257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:31:58.115257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:31:58.115257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:31:58.115257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:31:58.115257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:31:58.115257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:31:58.115257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:31:58.115257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:31:58.115257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:31:58.115257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:31:58.116553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:31:58.116553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:31:58.116553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:31:58.116553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:31:58.116553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:31:58.116553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:31:58.116553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:31:58.116553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:31:58.116553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:31:58.116553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:31:58.116553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:31:58.116553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:31:58.116553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:31:58.116553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:31:58.116553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:31:58.116553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:31:58.117459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:31:58.117459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:31:58.117459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:31:58.117459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:31:58.117459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:31:58.117459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:31:58.117459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:31:58.117459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:31:58.117459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:31:58.117459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:31:58.117459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:31:58.117459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:31:58.117459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 619/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:31:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:31:58.123108 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ade28-8903-4833-905a-1118376ef33d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:58.165796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:58.232225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ade28-8903-4833-905a-1118376ef33d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:58.273665 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c59770-3bd5-4c1d-ac0b-9cdbbb272b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:58.275957 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c59770-3bd5-4c1d-ac0b-9cdbbb272b00 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:58.275957 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c59770-3bd5-4c1d-ac0b-9cdbbb272b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:58.275957 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c59770-3bd5-4c1d-ac0b-9cdbbb272b00 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:58.275957 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c59770-3bd5-4c1d-ac0b-9cdbbb272b00 None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:31:58.276343 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c59770-3bd5-4c1d-ac0b-9cdbbb272b00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:58.277378 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c59770-3bd5-4c1d-ac0b-9cdbbb272b00 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:58.286827 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 590/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:31:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:31:58.292786 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ff51f-e4bf-4abc-b8c1-7f3ac4ebd4ca None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:58.352882 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:58.417557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ff51f-e4bf-4abc-b8c1-7f3ac4ebd4ca None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:58.459773 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0d834-0948-4e98-a61a-08e57daecb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:58.460677 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0d834-0948-4e98-a61a-08e57daecb8a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:58.461039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0d834-0948-4e98-a61a-08e57daecb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:58.461387 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0d834-0948-4e98-a61a-08e57daecb8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:58.475225 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0d834-0948-4e98-a61a-08e57daecb8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:58.476794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0d834-0948-4e98-a61a-08e57daecb8a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:58.495346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-93a0d834-0948-4e98-a61a-08e57daecb8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:31:58.495346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:31:58.495346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:31:58.495346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:31:58.495346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:31:58.495346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:31:58.495346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:31:58.495346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:31:58.495346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:31:58.495346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:31:58.495346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:31:58.495346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:31:58.495346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:31:58.495346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:31:58.495346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:31:58.496550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:31:58.496550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:31:58.496550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:31:58.496550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:31:58.496550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:31:58.496550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:31:58.496550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:31:58.496550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:31:58.496550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:31:58.496550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:31:58.496550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:31:58.496550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:31:58.496550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:31:58.496550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:31:58.496550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:31:58.496550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:31:58.496550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:31:58.497676 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:31:58.497676 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:31:58.497676 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:31:58.497676 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:31:58.498794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 620/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:31:58.505455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c59770-3bd5-4c1d-ac0b-9cdbbb272b00 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:58.627209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:58.631565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c59770-3bd5-4c1d-ac0b-9cdbbb272b00 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:58.678267 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32e5e3-105a-4bfa-a0d5-63f966c61395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:58.678948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32e5e3-105a-4bfa-a0d5-63f966c61395 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:58.679235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32e5e3-105a-4bfa-a0d5-63f966c61395 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:58.679555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32e5e3-105a-4bfa-a0d5-63f966c61395 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:58.680290 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32e5e3-105a-4bfa-a0d5-63f966c61395 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:31:58.680601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32e5e3-105a-4bfa-a0d5-63f966c61395 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:58.682004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32e5e3-105a-4bfa-a0d5-63f966c61395 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:58.698002 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 591/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:31:58.704454 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0d834-0948-4e98-a61a-08e57daecb8a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:58.833426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0d834-0948-4e98-a61a-08e57daecb8a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:58.836176 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:58.877041 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfdff9-ede2-4346-9693-35b4eee21a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:58.878083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfdff9-ede2-4346-9693-35b4eee21a6c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:58.878337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfdff9-ede2-4346-9693-35b4eee21a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:58.878663 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfdff9-ede2-4346-9693-35b4eee21a6c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:58.888556 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccfdff9-ede2-4346-9693-35b4eee21a6c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:58.890097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccfdff9-ede2-4346-9693-35b4eee21a6c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:58.908217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-4ccfdff9-ede2-4346-9693-35b4eee21a6c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:31:58.908217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:31:58.908217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:31:58.908217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:31:58.908217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:31:58.908217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:31:58.908217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:31:58.908217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:31:58.908217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:31:58.908217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:31:58.908217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:31:58.908217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:31:58.908217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:31:58.908217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:31:58.908217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:31:58.908217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:31:58.909333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:31:58.910348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:31:58.910348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:31:58.910348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:31:58.910348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:31:58.910348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:31:58.910348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:31:58.910348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:31:58.910348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:31:58.910348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:31:58.910348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:31:58.910348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 621/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:31:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:31:58.915626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32e5e3-105a-4bfa-a0d5-63f966c61395 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:59.038201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32e5e3-105a-4bfa-a0d5-63f966c61395 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:59.080482 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3289e-5ec7-4007-a6e1-ab3753deee53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:59.081314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3289e-5ec7-4007-a6e1-ab3753deee53 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:59.081671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3289e-5ec7-4007-a6e1-ab3753deee53 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:59.081945 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3289e-5ec7-4007-a6e1-ab3753deee53 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:59.082812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3289e-5ec7-4007-a6e1-ab3753deee53 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:31:59.083146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3289e-5ec7-4007-a6e1-ab3753deee53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:59.084568 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3289e-5ec7-4007-a6e1-ab3753deee53 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:59.106391 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 592/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Jul 12 13:31:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:31:59.112826 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfdff9-ede2-4346-9693-35b4eee21a6c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:59.234124 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfdff9-ede2-4346-9693-35b4eee21a6c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:59.278219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7aed05-375f-458f-9850-5ae82f3fd90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:59.278936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7aed05-375f-458f-9850-5ae82f3fd90b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:59.279328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7aed05-375f-458f-9850-5ae82f3fd90b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:59.279548 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7aed05-375f-458f-9850-5ae82f3fd90b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:59.314285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:59.364699 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7aed05-375f-458f-9850-5ae82f3fd90b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:59.366300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7aed05-375f-458f-9850-5ae82f3fd90b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:59.498565 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 622/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:31:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:31:59.507615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3289e-5ec7-4007-a6e1-ab3753deee53 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:59.529474 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:59.627750 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3289e-5ec7-4007-a6e1-ab3753deee53 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:59.669083 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d98d5-4088-4ad9-ad83-c08defe5f52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:59.669894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d98d5-4088-4ad9-ad83-c08defe5f52f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:59.670211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d98d5-4088-4ad9-ad83-c08defe5f52f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:59.670483 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d98d5-4088-4ad9-ad83-c08defe5f52f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:59.682879 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d98d5-4088-4ad9-ad83-c08defe5f52f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:59.684558 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d98d5-4088-4ad9-ad83-c08defe5f52f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:59.703978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-c14d98d5-4088-4ad9-ad83-c08defe5f52f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:31:59.703978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:31:59.703978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:31:59.703978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:31:59.703978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:31:59.703978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:31:59.703978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:31:59.703978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:31:59.703978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:31:59.703978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:31:59.703978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:31:59.703978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:31:59.703978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:31:59.703978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:31:59.703978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:31:59.705137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:31:59.705137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:31:59.705137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:31:59.705137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:31:59.705137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:31:59.705137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:31:59.705137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:31:59.705137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:31:59.705137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:31:59.705137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:31:59.705137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:31:59.705137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:31:59.705137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:31:59.705137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:31:59.705137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:31:59.705137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:31:59.705137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:31:59.711388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:31:59.711388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:31:59.711388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:31:59.711388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:31:59.711388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 593/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:31:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:31:59.713194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7aed05-375f-458f-9850-5ae82f3fd90b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:59.785761 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:59.834135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7aed05-375f-458f-9850-5ae82f3fd90b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:59.889289 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4b21d-211f-42ad-a43b-d178c518860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:59.893796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4b21d-211f-42ad-a43b-d178c518860a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:59.893796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4b21d-211f-42ad-a43b-d178c518860a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:59.893796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4b21d-211f-42ad-a43b-d178c518860a None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:59.893796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa4b21d-211f-42ad-a43b-d178c518860a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:31:59.893796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa4b21d-211f-42ad-a43b-d178c518860a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:59.893796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa4b21d-211f-42ad-a43b-d178c518860a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:59.907764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 623/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:31:59.919800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d98d5-4088-4ad9-ad83-c08defe5f52f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:00.004186 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:00.042484 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d98d5-4088-4ad9-ad83-c08defe5f52f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:00.090489 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64741841-2cdd-4490-86a7-f1c78589b206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:00.091419 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64741841-2cdd-4490-86a7-f1c78589b206 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:00.091676 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64741841-2cdd-4490-86a7-f1c78589b206 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:00.091954 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64741841-2cdd-4490-86a7-f1c78589b206 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:00.100162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64741841-2cdd-4490-86a7-f1c78589b206 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:00.101653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64741841-2cdd-4490-86a7-f1c78589b206 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:00.117825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-64741841-2cdd-4490-86a7-f1c78589b206 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:32:00.117825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:00.117825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:00.117825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:00.117825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:00.117825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:00.117825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:00.117825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:00.117825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:00.117825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:00.117825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:00.117825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:00.117825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:32:00.117825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:32:00.117825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:32:00.118840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:32:00.118840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:00.118840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:00.118840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:00.118840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:00.118840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:00.118840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:00.118840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:00.118840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:00.118840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:00.118840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:00.118840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:00.118840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:00.118840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:00.118840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:00.118840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:00.118840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:00.119738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:32:00.119738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:32:00.119738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:32:00.119738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:32:00.119738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:32:00.119738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:32:00.119738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:32:00.119738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:32:00.123962 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 594/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:31:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:00.125565 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4b21d-211f-42ad-a43b-d178c518860a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:00.260424 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4b21d-211f-42ad-a43b-d178c518860a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:00.288951 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:00.301794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9e5f7-35af-433d-aff1-8367fba8db96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:00.302543 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9e5f7-35af-433d-aff1-8367fba8db96 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:00.302927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9e5f7-35af-433d-aff1-8367fba8db96 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:00.303298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9e5f7-35af-433d-aff1-8367fba8db96 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:00.304177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9e5f7-35af-433d-aff1-8367fba8db96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:00.304666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9e5f7-35af-433d-aff1-8367fba8db96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:00.306211 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9e5f7-35af-433d-aff1-8367fba8db96 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:00.325035 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 624/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:32:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:00.326889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64741841-2cdd-4490-86a7-f1c78589b206 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:00.444098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64741841-2cdd-4490-86a7-f1c78589b206 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:00.474956 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:00.482611 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a13ea7-815e-47d0-91ca-e0eb2e330183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:00.483279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a13ea7-815e-47d0-91ca-e0eb2e330183 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:00.483569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a13ea7-815e-47d0-91ca-e0eb2e330183 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:00.483845 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a13ea7-815e-47d0-91ca-e0eb2e330183 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:00.484715 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a13ea7-815e-47d0-91ca-e0eb2e330183 None admin] RBAC: Adding query filter params (role.id=057d9335b3b1412ebfb6bfb27f02f96a, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=f1308deea1ea4629a1ac22d4b903ad4f) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:00.485025 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a13ea7-815e-47d0-91ca-e0eb2e330183 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:00.486661 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a13ea7-815e-47d0-91ca-e0eb2e330183 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:00.493263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 595/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:32:00 2022] GET /identity/v3/role_assignments?user.id=f1308deea1ea4629a1ac22d4b903ad4f&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=057d9335b3b1412ebfb6bfb27f02f96a => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:02.593346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02216b2-7257-4045-82b4-ff72416b25c7 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:02.593934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02216b2-7257-4045-82b4-ff72416b25c7 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:02.593934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02216b2-7257-4045-82b4-ff72416b25c7 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:02.595284 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 625/1220] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:32:02.601757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f4448-992f-4ec9-bfaa-fd52f743096d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:02.602245 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f4448-992f-4ec9-bfaa-fd52f743096d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:02.602371 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f4448-992f-4ec9-bfaa-fd52f743096d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:02.622423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b85f4448-992f-4ec9-bfaa-fd52f743096d None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:32:02.623727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b85f4448-992f-4ec9-bfaa-fd52f743096d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:02.714037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 596/1221] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:32:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:32:02.737464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02216b2-7257-4045-82b4-ff72416b25c7 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:02.844740 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02216b2-7257-4045-82b4-ff72416b25c7 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:02.882889 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed151f-af09-48ad-9bc0-20e8c6825e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:02.883724 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed151f-af09-48ad-9bc0-20e8c6825e98 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:02.884004 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed151f-af09-48ad-9bc0-20e8c6825e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:02.884300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed151f-af09-48ad-9bc0-20e8c6825e98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:02.894994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed151f-af09-48ad-9bc0-20e8c6825e98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:32:02.903802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-02ed151f-af09-48ad-9bc0-20e8c6825e98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:32:02.903802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:02.903802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:02.903802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:02.903802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:02.903802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:02.903802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:02.903802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:02.903802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:02.903802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:02.903802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:02.903802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:02.903802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:32:02.903802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:32:02.903802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:32:02.903802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:32:02.904950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:32:02.904950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:32:02.904950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:32:02.904950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:32:02.904950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:02.904950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:02.904950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:02.904950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:02.904950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:02.904950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:02.904950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:02.904950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:02.904950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:02.904950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:02.904950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:02.904950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:02.906010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:02.906010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:02.906010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:02.906010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:02.906010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:32:02.906010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:32:02.906010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:32:02.906010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:32:02.906010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:32:02.906010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:32:02.906010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:32:02.906010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:02.906010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 626/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:32:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:02.914021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f4448-992f-4ec9-bfaa-fd52f743096d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:03.015248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f4448-992f-4ec9-bfaa-fd52f743096d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:03.039272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:03.050528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae952844-466c-4a20-9507-f499c029d026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:03.051168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae952844-466c-4a20-9507-f499c029d026 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:03.051526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae952844-466c-4a20-9507-f499c029d026 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:03.051812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae952844-466c-4a20-9507-f499c029d026 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:03.052589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae952844-466c-4a20-9507-f499c029d026 None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:03.052889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae952844-466c-4a20-9507-f499c029d026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:03.054341 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae952844-466c-4a20-9507-f499c029d026 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:03.063965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 597/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:32:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:03.071208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed151f-af09-48ad-9bc0-20e8c6825e98 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:03.121223 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:03.182790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed151f-af09-48ad-9bc0-20e8c6825e98 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:03.219794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6200c4-c0f5-4ffa-b432-216c88bef520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:03.220597 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6200c4-c0f5-4ffa-b432-216c88bef520 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:03.220883 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6200c4-c0f5-4ffa-b432-216c88bef520 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:03.221157 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6200c4-c0f5-4ffa-b432-216c88bef520 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:03.233734 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6200c4-c0f5-4ffa-b432-216c88bef520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:03.235418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6200c4-c0f5-4ffa-b432-216c88bef520 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:03.256380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-ec6200c4-c0f5-4ffa-b432-216c88bef520 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:03.256380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:03.256380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:03.256380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:03.256380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:03.256380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:03.256380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:03.256380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:03.256380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:03.256380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:03.256380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:03.256380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:03.256380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:32:03.256380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:32:03.256380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:32:03.257553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:32:03.257553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:03.257553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:03.257553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:03.257553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:03.257553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:03.257553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:03.257553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:03.257553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:03.257553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:03.257553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:03.257553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:03.257553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:03.257553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:03.257553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:03.257553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:03.257553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:03.258861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:32:03.258861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:32:03.258861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:03.258861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:03.258861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 627/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:32:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:03.264543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae952844-466c-4a20-9507-f499c029d026 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:03.377566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae952844-466c-4a20-9507-f499c029d026 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:03.414903 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd30b20-6441-4f06-8db6-3eaca7381458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:03.415571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd30b20-6441-4f06-8db6-3eaca7381458 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:03.415859 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd30b20-6441-4f06-8db6-3eaca7381458 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:03.416137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd30b20-6441-4f06-8db6-3eaca7381458 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:03.416875 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd30b20-6441-4f06-8db6-3eaca7381458 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:03.417181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd30b20-6441-4f06-8db6-3eaca7381458 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:03.418609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd30b20-6441-4f06-8db6-3eaca7381458 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:03.433242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 598/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:03.439056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6200c4-c0f5-4ffa-b432-216c88bef520 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:03.559570 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:03.566751 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6200c4-c0f5-4ffa-b432-216c88bef520 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:03.600328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9a104-7369-41be-86e0-eae1bfbd1154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:03.601123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9a104-7369-41be-86e0-eae1bfbd1154 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:03.601403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9a104-7369-41be-86e0-eae1bfbd1154 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:03.601674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9a104-7369-41be-86e0-eae1bfbd1154 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:03.608890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9a104-7369-41be-86e0-eae1bfbd1154 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:03.610385 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9a104-7369-41be-86e0-eae1bfbd1154 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:03.628042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-d1b9a104-7369-41be-86e0-eae1bfbd1154 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:32:03.628042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:03.628042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:03.628042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:03.628042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:03.628042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:03.628042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:03.628042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:03.628042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:03.628042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:03.628042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:03.628042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:03.628042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:32:03.628042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:32:03.628042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:32:03.628042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:03.629205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:03.630979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:03.630979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:03.630979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:32:03.630979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:32:03.630979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:32:03.630979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:32:03.630979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:32:03.630979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:32:03.630979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:32:03.630979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:03.631727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 628/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:32:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:03.637494 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd30b20-6441-4f06-8db6-3eaca7381458 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:03.690219 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:03.749249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd30b20-6441-4f06-8db6-3eaca7381458 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:03.784540 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f6ca9-5e6d-438b-9e42-263de9cc4444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:03.785222 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f6ca9-5e6d-438b-9e42-263de9cc4444 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:03.785600 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f6ca9-5e6d-438b-9e42-263de9cc4444 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:03.786379 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f6ca9-5e6d-438b-9e42-263de9cc4444 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:03.787312 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f6ca9-5e6d-438b-9e42-263de9cc4444 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:03.787595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f6ca9-5e6d-438b-9e42-263de9cc4444 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:03.789062 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f6ca9-5e6d-438b-9e42-263de9cc4444 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:03.810947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 599/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Jul 12 13:32:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:03.818030 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9a104-7369-41be-86e0-eae1bfbd1154 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:03.936806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9a104-7369-41be-86e0-eae1bfbd1154 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:03.979608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cfb63-08e7-4545-8fcb-337505a041e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:03.980701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cfb63-08e7-4545-8fcb-337505a041e0 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:03.981066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cfb63-08e7-4545-8fcb-337505a041e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:03.981453 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cfb63-08e7-4545-8fcb-337505a041e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:03.993163 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4cfb63-08e7-4545-8fcb-337505a041e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:03.994765 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4cfb63-08e7-4545-8fcb-337505a041e0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:04.017857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-0f4cfb63-08e7-4545-8fcb-337505a041e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:04.017857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:04.017857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:04.017857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:04.017857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:04.017857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:04.017857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:04.017857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:04.017857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:04.017857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:04.017857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:04.017857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:04.017857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:32:04.017857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:32:04.017857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:32:04.018962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:32:04.018962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:04.018962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:04.018962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:04.018962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:04.018962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:04.018962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:04.018962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:04.018962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:04.018962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:04.018962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:04.018962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:04.018962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:04.018962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:04.018962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:04.018962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:04.018962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:04.019879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:32:04.019879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:32:04.019879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:04.019879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:04.019879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 629/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:32:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:04.025541 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f6ca9-5e6d-438b-9e42-263de9cc4444 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:04.136092 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f6ca9-5e6d-438b-9e42-263de9cc4444 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:04.138434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:04.172462 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a02c93-5f80-4256-b33d-73a0a465a73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:04.173188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a02c93-5f80-4256-b33d-73a0a465a73e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:04.173514 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a02c93-5f80-4256-b33d-73a0a465a73e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:04.173849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a02c93-5f80-4256-b33d-73a0a465a73e None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:04.175076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a02c93-5f80-4256-b33d-73a0a465a73e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:04.175421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a02c93-5f80-4256-b33d-73a0a465a73e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:04.176976 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a02c93-5f80-4256-b33d-73a0a465a73e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:04.189064 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 600/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:04.194573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cfb63-08e7-4545-8fcb-337505a041e0 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:04.204608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:04.312914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cfb63-08e7-4545-8fcb-337505a041e0 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:04.354420 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5a3d7-b91f-4a8b-ad65-b4056540bb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:04.355283 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5a3d7-b91f-4a8b-ad65-b4056540bb71 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:04.355601 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5a3d7-b91f-4a8b-ad65-b4056540bb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:04.355888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5a3d7-b91f-4a8b-ad65-b4056540bb71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:04.364320 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5a3d7-b91f-4a8b-ad65-b4056540bb71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:04.365831 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5a3d7-b91f-4a8b-ad65-b4056540bb71 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:04.381594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-1bf5a3d7-b91f-4a8b-ad65-b4056540bb71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:32:04.381594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:04.381594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:04.381594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:04.381594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:04.381594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:04.381594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:04.381594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:04.381594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:04.381594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:04.381594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:04.381594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:04.381594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:32:04.381594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:32:04.381594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:32:04.382708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:32:04.382708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:04.382708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:04.382708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:04.382708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:04.382708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:04.382708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:04.382708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:04.382708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:04.382708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:04.382708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:04.382708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:04.382708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:04.382708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:04.382708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:04.382708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:04.382708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:04.383680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:32:04.383680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:32:04.383680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:32:04.383680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:32:04.383680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:32:04.383680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:32:04.383680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:32:04.383680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:04.383680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 630/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:32:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:04.389129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a02c93-5f80-4256-b33d-73a0a465a73e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:04.495498 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a02c93-5f80-4256-b33d-73a0a465a73e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:04.535203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76ff3e-7d16-490c-b01f-f555b7f56a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:04.535865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76ff3e-7d16-490c-b01f-f555b7f56a0d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:04.536149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76ff3e-7d16-490c-b01f-f555b7f56a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:04.536427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76ff3e-7d16-490c-b01f-f555b7f56a0d None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:04.537212 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76ff3e-7d16-490c-b01f-f555b7f56a0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:04.537522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76ff3e-7d16-490c-b01f-f555b7f56a0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:04.539041 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76ff3e-7d16-490c-b01f-f555b7f56a0d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:04.554835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 601/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:32:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:04.561116 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5a3d7-b91f-4a8b-ad65-b4056540bb71 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:04.640366 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:04.680503 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5a3d7-b91f-4a8b-ad65-b4056540bb71 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:04.719068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0292823-7477-44fa-8621-23f7c0d328bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:04.719965 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0292823-7477-44fa-8621-23f7c0d328bc None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:04.720251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0292823-7477-44fa-8621-23f7c0d328bc None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:04.720535 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0292823-7477-44fa-8621-23f7c0d328bc None admin] PATH_INFO: `/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/f1308deea1ea4629a1ac22d4b903ad4f/roles/057d9335b3b1412ebfb6bfb27f02f96a` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:04.737021 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0292823-7477-44fa-8621-23f7c0d328bc None admin] RBAC: Authorizing `identity:create_grant(project_id=36aa0326fd0545dface07a9ddc308ccd, user_id=f1308deea1ea4629a1ac22d4b903ad4f, role_id=057d9335b3b1412ebfb6bfb27f02f96a)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:04.738693 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0292823-7477-44fa-8621-23f7c0d328bc None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:04.753076 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 631/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:32:04 2022] PUT /identity/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/f1308deea1ea4629a1ac22d4b903ad4f/roles/057d9335b3b1412ebfb6bfb27f02f96a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:32:06.965134 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816d2b4-b858-4b90-b67b-f225c6f4ab21 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:06.965681 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816d2b4-b858-4b90-b67b-f225c6f4ab21 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:06.965681 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816d2b4-b858-4b90-b67b-f225c6f4ab21 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:06.967256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 602/1233] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:32:06.972920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9f60c-c1b1-4ba6-8eca-34f724d70473 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:06.973343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9f60c-c1b1-4ba6-8eca-34f724d70473 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:06.973484 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9f60c-c1b1-4ba6-8eca-34f724d70473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:06.994917 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8fe9f60c-c1b1-4ba6-8eca-34f724d70473 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:32:06.996109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8fe9f60c-c1b1-4ba6-8eca-34f724d70473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:07.067632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 632/1234] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:32:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:32:07.092222 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816d2b4-b858-4b90-b67b-f225c6f4ab21 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:07.119407 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:07.204839 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816d2b4-b858-4b90-b67b-f225c6f4ab21 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:07.243473 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035cc45-aab1-4350-8f88-e440a727914d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:07.244619 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035cc45-aab1-4350-8f88-e440a727914d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:07.244932 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035cc45-aab1-4350-8f88-e440a727914d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:07.245325 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035cc45-aab1-4350-8f88-e440a727914d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:07.257647 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035cc45-aab1-4350-8f88-e440a727914d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:32:07.265863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-7035cc45-aab1-4350-8f88-e440a727914d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:32:07.265863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:07.265863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:07.265863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:07.265863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:07.265863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:07.265863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:07.265863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:07.265863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:07.265863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:07.265863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:07.265863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:07.265863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:32:07.265863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:32:07.265863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:32:07.265863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:32:07.266898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:32:07.266898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:32:07.266898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:32:07.266898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:32:07.266898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:07.266898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:07.266898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:07.266898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:07.266898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:07.266898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:07.266898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:07.266898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:07.266898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:07.266898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:07.266898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:07.266898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:07.268010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:07.268010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:07.268010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:07.268010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:07.268010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:32:07.268010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:32:07.268010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:32:07.268010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:32:07.268010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:32:07.268010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:32:07.268010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:32:07.268010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:32:07.268010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 603/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jul 12 13:32:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:07.273274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9f60c-c1b1-4ba6-8eca-34f724d70473 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:07.326324 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:07.384978 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9f60c-c1b1-4ba6-8eca-34f724d70473 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:07.421060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daaab9b-237c-4a01-b96c-5c80a984da07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:07.421772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daaab9b-237c-4a01-b96c-5c80a984da07 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:07.422152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daaab9b-237c-4a01-b96c-5c80a984da07 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:07.422429 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daaab9b-237c-4a01-b96c-5c80a984da07 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:07.423278 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daaab9b-237c-4a01-b96c-5c80a984da07 None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:07.423522 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daaab9b-237c-4a01-b96c-5c80a984da07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:07.425024 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daaab9b-237c-4a01-b96c-5c80a984da07 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:07.434372 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 633/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jul 12 13:32:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:07.440355 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035cc45-aab1-4350-8f88-e440a727914d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:07.542148 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:07.566785 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035cc45-aab1-4350-8f88-e440a727914d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:07.604834 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463f788-c055-4ae3-8382-03f4762f0827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:07.605685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463f788-c055-4ae3-8382-03f4762f0827 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:07.606022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463f788-c055-4ae3-8382-03f4762f0827 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:07.606305 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463f788-c055-4ae3-8382-03f4762f0827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:07.619258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463f788-c055-4ae3-8382-03f4762f0827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:07.620719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463f788-c055-4ae3-8382-03f4762f0827 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:07.639833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-4463f788-c055-4ae3-8382-03f4762f0827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:07.639833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:07.639833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:07.639833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:07.639833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:07.639833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:07.639833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:07.639833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:07.639833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:07.639833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:07.639833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:07.639833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:07.639833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:32:07.639833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:32:07.639833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:32:07.640849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:32:07.640849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:07.640849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:07.640849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:07.640849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:07.640849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:07.640849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:07.640849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:07.640849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:07.640849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:07.640849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:07.640849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:07.640849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:07.640849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:07.640849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:07.640849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:07.640849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:07.641795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:32:07.641795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:32:07.641795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:07.641795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:32:07.641795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 604/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:32:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:07.647298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daaab9b-237c-4a01-b96c-5c80a984da07 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:07.759760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daaab9b-237c-4a01-b96c-5c80a984da07 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:07.769844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:07.799416 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec82bb-1a90-4421-b685-414a48c333b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:07.800313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec82bb-1a90-4421-b685-414a48c333b5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:07.800628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec82bb-1a90-4421-b685-414a48c333b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:07.800924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec82bb-1a90-4421-b685-414a48c333b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:07.802382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec82bb-1a90-4421-b685-414a48c333b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:07.802692 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec82bb-1a90-4421-b685-414a48c333b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:07.804298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec82bb-1a90-4421-b685-414a48c333b5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:07.817816 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 634/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:07.824124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463f788-c055-4ae3-8382-03f4762f0827 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:07.940693 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463f788-c055-4ae3-8382-03f4762f0827 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:07.978677 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443477fd-ac95-451e-b382-8ac25bb59334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:07.979472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443477fd-ac95-451e-b382-8ac25bb59334 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:07.979755 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443477fd-ac95-451e-b382-8ac25bb59334 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:07.980033 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443477fd-ac95-451e-b382-8ac25bb59334 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:07.983267 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:07.988935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443477fd-ac95-451e-b382-8ac25bb59334 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:07.990941 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443477fd-ac95-451e-b382-8ac25bb59334 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:08.007174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-443477fd-ac95-451e-b382-8ac25bb59334 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:32:08.007174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:08.007174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:08.007174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:08.007174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:08.007174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:08.007174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:08.007174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:08.007174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:08.007174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:08.007174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:08.007174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:08.007174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:32:08.007174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:32:08.007174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:32:08.007174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:08.013418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:08.014694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:08.014694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:08.014694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:32:08.014694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:32:08.014694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:32:08.014694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:32:08.014694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:32:08.014694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:32:08.014694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:32:08.014694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:32:08.014694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 605/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:32:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:08.015405 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec82bb-1a90-4421-b685-414a48c333b5 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:08.131251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec82bb-1a90-4421-b685-414a48c333b5 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:08.168307 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b22c7-0cbf-4a0b-8627-d9b2506ace59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:08.168985 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b22c7-0cbf-4a0b-8627-d9b2506ace59 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:08.169271 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b22c7-0cbf-4a0b-8627-d9b2506ace59 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:08.169549 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b22c7-0cbf-4a0b-8627-d9b2506ace59 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:08.170422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b22c7-0cbf-4a0b-8627-d9b2506ace59 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:08.170745 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b22c7-0cbf-4a0b-8627-d9b2506ace59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:08.172382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b22c7-0cbf-4a0b-8627-d9b2506ace59 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:08.191136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 635/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Jul 12 13:32:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:32:08.196918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443477fd-ac95-451e-b382-8ac25bb59334 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:08.308896 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443477fd-ac95-451e-b382-8ac25bb59334 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:08.344884 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e354d60-5ac2-40ac-bf02-1476635bd4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:08.345545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e354d60-5ac2-40ac-bf02-1476635bd4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:08.345822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e354d60-5ac2-40ac-bf02-1476635bd4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:08.346126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e354d60-5ac2-40ac-bf02-1476635bd4f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:08.418737 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:08.426181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e354d60-5ac2-40ac-bf02-1476635bd4f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:08.427651 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e354d60-5ac2-40ac-bf02-1476635bd4f5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:08.535990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 606/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:32:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:32:08.541842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b22c7-0cbf-4a0b-8627-d9b2506ace59 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:08.582462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:08.665217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b22c7-0cbf-4a0b-8627-d9b2506ace59 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:08.703324 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83bd8ad-01e6-4843-abe8-11371ebc966f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:08.704108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bd8ad-01e6-4843-abe8-11371ebc966f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:08.704395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bd8ad-01e6-4843-abe8-11371ebc966f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:08.704675 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bd8ad-01e6-4843-abe8-11371ebc966f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:08.715562 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83bd8ad-01e6-4843-abe8-11371ebc966f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:08.717074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83bd8ad-01e6-4843-abe8-11371ebc966f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:08.737107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-a83bd8ad-01e6-4843-abe8-11371ebc966f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:08.737107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:08.737107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:08.737107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:08.737107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:08.737107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:08.737107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:08.737107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:08.737107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:08.737107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:08.737107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:08.737107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:08.737107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:32:08.737107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:32:08.737107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:32:08.738156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:32:08.738156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:08.738156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:08.738156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:08.738156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:08.738156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:08.738156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:08.738156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:08.738156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:08.738156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:08.738156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:08.738156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:08.738156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:08.738156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:08.738156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:08.738156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:08.738156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:08.739572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:32:08.739572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:32:08.739572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:08.739572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:08.739572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 636/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:32:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:08.744645 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e354d60-5ac2-40ac-bf02-1476635bd4f5 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:08.847943 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:08.858981 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e354d60-5ac2-40ac-bf02-1476635bd4f5 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:08.897811 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7b462-5c06-469d-b884-df6c856211cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:08.898402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7b462-5c06-469d-b884-df6c856211cb None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:08.898684 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7b462-5c06-469d-b884-df6c856211cb None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:08.898960 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7b462-5c06-469d-b884-df6c856211cb None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:08.899681 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7b462-5c06-469d-b884-df6c856211cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:08.899997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7b462-5c06-469d-b884-df6c856211cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:08.901363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7b462-5c06-469d-b884-df6c856211cb None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:08.913982 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 607/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:08.919648 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83bd8ad-01e6-4843-abe8-11371ebc966f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:09.004892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:09.027653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83bd8ad-01e6-4843-abe8-11371ebc966f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:09.063676 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b879a-8380-4ae8-9aab-d1699aeb2271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:09.064503 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b879a-8380-4ae8-9aab-d1699aeb2271 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:09.064824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b879a-8380-4ae8-9aab-d1699aeb2271 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:09.065056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b879a-8380-4ae8-9aab-d1699aeb2271 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:09.072176 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b879a-8380-4ae8-9aab-d1699aeb2271 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:09.073663 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b879a-8380-4ae8-9aab-d1699aeb2271 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:09.088047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-417b879a-8380-4ae8-9aab-d1699aeb2271 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:32:09.088047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:09.088047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:09.088047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:09.088047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:09.088047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:09.088047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:09.088047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:09.088047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:09.088047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:09.088047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:09.088047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:09.088047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:32:09.088047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:32:09.088047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:32:09.088991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:32:09.088991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:09.088991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:09.088991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:09.088991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:09.088991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:09.088991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:09.088991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:09.088991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:09.088991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:09.088991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:09.088991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:09.088991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:09.088991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:09.088991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:09.088991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:09.088991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:09.089895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:32:09.089895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:32:09.089895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:32:09.089895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:32:09.089895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:32:09.089895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:32:09.089895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:32:09.089895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:09.089895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 637/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:32:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:09.095235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7b462-5c06-469d-b884-df6c856211cb None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:09.208107 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7b462-5c06-469d-b884-df6c856211cb None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:09.242483 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2a419-5351-4357-b9ad-a9a05692641f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:09.243148 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2a419-5351-4357-b9ad-a9a05692641f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:09.243435 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2a419-5351-4357-b9ad-a9a05692641f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:09.243759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2a419-5351-4357-b9ad-a9a05692641f None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:09.244543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2a419-5351-4357-b9ad-a9a05692641f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:09.244862 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2a419-5351-4357-b9ad-a9a05692641f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:09.246321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2a419-5351-4357-b9ad-a9a05692641f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:09.258939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 608/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:32:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:09.265263 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b879a-8380-4ae8-9aab-d1699aeb2271 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:09.373297 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b879a-8380-4ae8-9aab-d1699aeb2271 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:09.411307 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea3cbe-dddb-4e4a-aec6-f9617352d609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:09.412275 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea3cbe-dddb-4e4a-aec6-f9617352d609 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:09.412792 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea3cbe-dddb-4e4a-aec6-f9617352d609 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:09.413150 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea3cbe-dddb-4e4a-aec6-f9617352d609 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:09.414405 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea3cbe-dddb-4e4a-aec6-f9617352d609 None admin] RBAC: Adding query filter params (role.id=057d9335b3b1412ebfb6bfb27f02f96a, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=f1308deea1ea4629a1ac22d4b903ad4f) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:09.414882 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea3cbe-dddb-4e4a-aec6-f9617352d609 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:09.416895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea3cbe-dddb-4e4a-aec6-f9617352d609 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:09.441990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 638/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:32:09 2022] GET /identity/v3/role_assignments?user.id=f1308deea1ea4629a1ac22d4b903ad4f&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=057d9335b3b1412ebfb6bfb27f02f96a => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:11.872021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92f060-29ef-49b6-8c61-4cc70f4143a2 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:11.872571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92f060-29ef-49b6-8c61-4cc70f4143a2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:11.873132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92f060-29ef-49b6-8c61-4cc70f4143a2 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:11.875152 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 609/1247] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:32:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:32:11.882264 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9888d-9181-4dd6-bd21-fb0e3fee7739 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:11.883191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9888d-9181-4dd6-bd21-fb0e3fee7739 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:11.883191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9888d-9181-4dd6-bd21-fb0e3fee7739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:11.888240 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:11.904039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-57b9888d-9181-4dd6-bd21-fb0e3fee7739 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:32:11.905716 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-57b9888d-9181-4dd6-bd21-fb0e3fee7739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:11.989481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 639/1248] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:32:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:32:12.014442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92f060-29ef-49b6-8c61-4cc70f4143a2 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:12.036006 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:12.137010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92f060-29ef-49b6-8c61-4cc70f4143a2 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:12.173543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c3667-9716-4c5c-a291-c0722bfc129d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:12.174361 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c3667-9716-4c5c-a291-c0722bfc129d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:12.174650 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c3667-9716-4c5c-a291-c0722bfc129d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:12.174925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c3667-9716-4c5c-a291-c0722bfc129d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:12.186979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c3667-9716-4c5c-a291-c0722bfc129d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:32:12.195249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-469c3667-9716-4c5c-a291-c0722bfc129d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:32:12.195249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:12.195249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:12.195249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:12.195249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:12.195249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:12.195249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:12.195249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:12.195249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:12.195249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:12.195249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:12.195249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:12.195249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:32:12.195249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:32:12.195249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:32:12.195249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:32:12.196819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:32:12.196819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:32:12.196819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:32:12.196819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:32:12.196819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:12.196819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:12.196819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:12.196819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:12.196819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:12.196819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:12.196819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:12.196819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:12.196819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:12.196819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:12.196819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:12.196819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:12.197906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:12.197906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:12.197906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:12.197906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:12.197906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:32:12.197906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:32:12.197906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:32:12.197906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:32:12.197906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:32:12.197906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:32:12.197906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:32:12.197906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:32:12.197906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 610/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:32:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:12.203457 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9888d-9181-4dd6-bd21-fb0e3fee7739 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:12.315409 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9888d-9181-4dd6-bd21-fb0e3fee7739 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:12.352081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:12.358345 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db015e9-d535-4927-9123-2ac26c0926e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:12.359164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db015e9-d535-4927-9123-2ac26c0926e9 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:12.359530 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db015e9-d535-4927-9123-2ac26c0926e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:12.359851 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db015e9-d535-4927-9123-2ac26c0926e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:12.360829 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db015e9-d535-4927-9123-2ac26c0926e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:12.361278 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db015e9-d535-4927-9123-2ac26c0926e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:12.362970 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db015e9-d535-4927-9123-2ac26c0926e9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:12.373865 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 640/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:32:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:12.380335 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c3667-9716-4c5c-a291-c0722bfc129d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:12.466372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:12.491579 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c3667-9716-4c5c-a291-c0722bfc129d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:12.531273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba8ccb-51f5-41c6-a38b-ee7f0966b1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:12.532305 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba8ccb-51f5-41c6-a38b-ee7f0966b1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:12.532787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba8ccb-51f5-41c6-a38b-ee7f0966b1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:12.533457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba8ccb-51f5-41c6-a38b-ee7f0966b1f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:12.554733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba8ccb-51f5-41c6-a38b-ee7f0966b1f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:12.556621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba8ccb-51f5-41c6-a38b-ee7f0966b1f9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:12.585641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-99ba8ccb-51f5-41c6-a38b-ee7f0966b1f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:12.585641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:12.585641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:12.585641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:12.585641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:12.585641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:12.585641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:12.585641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:12.585641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:12.585641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:12.585641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:12.585641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:12.585641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:32:12.585641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:32:12.585641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:32:12.586918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:32:12.586918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:12.586918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:12.586918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:12.586918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:12.586918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:12.586918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:12.586918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:12.586918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:12.586918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:12.586918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:12.586918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:12.586918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:12.586918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:12.586918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:12.586918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:12.586918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:12.588148 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:32:12.588148 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:32:12.588148 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:12.588148 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:32:12.588148 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 611/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:32:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:12.595613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db015e9-d535-4927-9123-2ac26c0926e9 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:12.720230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db015e9-d535-4927-9123-2ac26c0926e9 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:12.757923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b4842-90ac-4dcf-a1ac-8157f5d8b131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:12.758633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b4842-90ac-4dcf-a1ac-8157f5d8b131 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:12.758909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b4842-90ac-4dcf-a1ac-8157f5d8b131 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:12.759202 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b4842-90ac-4dcf-a1ac-8157f5d8b131 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:12.759906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b4842-90ac-4dcf-a1ac-8157f5d8b131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:12.760214 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b4842-90ac-4dcf-a1ac-8157f5d8b131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:12.761592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b4842-90ac-4dcf-a1ac-8157f5d8b131 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:12.774107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 641/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:12.780279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba8ccb-51f5-41c6-a38b-ee7f0966b1f9 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:12.903318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba8ccb-51f5-41c6-a38b-ee7f0966b1f9 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:12.932365 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:12.942912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8327570-b78c-429a-8e67-1958be9885d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:12.943898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8327570-b78c-429a-8e67-1958be9885d9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:12.944300 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8327570-b78c-429a-8e67-1958be9885d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:12.944726 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8327570-b78c-429a-8e67-1958be9885d9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:12.954055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8327570-b78c-429a-8e67-1958be9885d9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:12.955718 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8327570-b78c-429a-8e67-1958be9885d9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:12.971272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-b8327570-b78c-429a-8e67-1958be9885d9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:32:12.971272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:12.971272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:12.971272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:12.971272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:12.971272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:12.971272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:12.971272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:12.971272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:12.971272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:12.971272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:12.971272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:12.971272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:32:12.971272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:32:12.971272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:32:12.971272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:12.972262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:12.974220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:12.974220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:12.974220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:32:12.974220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:32:12.974220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:32:12.974220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:32:12.974220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:32:12.974220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:32:12.974220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:32:12.974220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:32:12.974220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 612/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:32:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:12.978679 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b4842-90ac-4dcf-a1ac-8157f5d8b131 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:13.044608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:13.095989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b4842-90ac-4dcf-a1ac-8157f5d8b131 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:13.134382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce51f13-9302-48b2-816a-3ea776546acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:13.135056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce51f13-9302-48b2-816a-3ea776546acd None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:13.135338 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce51f13-9302-48b2-816a-3ea776546acd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:13.135613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce51f13-9302-48b2-816a-3ea776546acd None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:13.136447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce51f13-9302-48b2-816a-3ea776546acd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:13.136759 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce51f13-9302-48b2-816a-3ea776546acd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:13.138222 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce51f13-9302-48b2-816a-3ea776546acd None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:13.156036 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 642/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Jul 12 13:32:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:32:13.162669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8327570-b78c-429a-8e67-1958be9885d9 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:13.286670 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8327570-b78c-429a-8e67-1958be9885d9 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:13.323016 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4f95e-687c-4910-a643-e2e7e892cc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:13.323644 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4f95e-687c-4910-a643-e2e7e892cc04 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:13.323929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4f95e-687c-4910-a643-e2e7e892cc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:13.324212 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4f95e-687c-4910-a643-e2e7e892cc04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:13.392763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:13.411050 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4f95e-687c-4910-a643-e2e7e892cc04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:13.413207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4f95e-687c-4910-a643-e2e7e892cc04 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:13.528114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 613/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:32:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:32:13.534441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce51f13-9302-48b2-816a-3ea776546acd None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:13.652735 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce51f13-9302-48b2-816a-3ea776546acd None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:13.665996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:13.689294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03810cad-0873-477f-963e-da11968f649c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:13.690191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03810cad-0873-477f-963e-da11968f649c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:13.690521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03810cad-0873-477f-963e-da11968f649c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:13.690836 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03810cad-0873-477f-963e-da11968f649c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:13.702677 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03810cad-0873-477f-963e-da11968f649c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:13.704222 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03810cad-0873-477f-963e-da11968f649c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:13.724123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-03810cad-0873-477f-963e-da11968f649c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:13.724123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:13.724123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:13.724123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:13.724123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:13.724123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:13.724123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:13.724123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:13.724123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:13.724123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:13.724123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:13.724123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:13.724123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:32:13.724123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:32:13.724123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:32:13.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:32:13.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:13.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:13.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:13.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:13.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:13.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:13.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:13.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:13.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:13.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:13.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:13.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:13.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:13.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:13.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:13.725130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:13.726072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:32:13.726072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:32:13.726072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:13.726072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:13.726072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 643/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:32:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:13.731643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4f95e-687c-4910-a643-e2e7e892cc04 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:13.826144 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:13.839853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4f95e-687c-4910-a643-e2e7e892cc04 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:13.882830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670aa8a-2669-4880-90af-00c799f7b8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:13.883542 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670aa8a-2669-4880-90af-00c799f7b8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:13.883925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670aa8a-2669-4880-90af-00c799f7b8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:13.884202 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670aa8a-2669-4880-90af-00c799f7b8b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:13.885001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670aa8a-2669-4880-90af-00c799f7b8b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:13.885397 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670aa8a-2669-4880-90af-00c799f7b8b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:13.886948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670aa8a-2669-4880-90af-00c799f7b8b4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:13.900125 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 614/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:13.905924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03810cad-0873-477f-963e-da11968f649c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:14.015107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03810cad-0873-477f-963e-da11968f649c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:14.052328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d43818-b097-4a94-9a54-caffbf1e24fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:14.053158 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d43818-b097-4a94-9a54-caffbf1e24fc None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:14.053509 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d43818-b097-4a94-9a54-caffbf1e24fc None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:14.053801 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d43818-b097-4a94-9a54-caffbf1e24fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:14.061845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d43818-b097-4a94-9a54-caffbf1e24fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:14.063536 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d43818-b097-4a94-9a54-caffbf1e24fc None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:14.078628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-31d43818-b097-4a94-9a54-caffbf1e24fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:32:14.078628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:14.078628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:14.078628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:14.078628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:14.078628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:14.078628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:14.078628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:14.078628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:14.078628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:14.078628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:14.078628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:14.078628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:32:14.078628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:32:14.078628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:32:14.079924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:32:14.079924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:14.079924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:14.079924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:14.079924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:14.079924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:14.079924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:14.079924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:14.079924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:14.079924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:14.079924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:14.079924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:14.079924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:14.079924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:14.079924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:14.079924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:14.079924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:14.081563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:32:14.081563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:32:14.081563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:32:14.081563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:32:14.081563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:32:14.081563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:32:14.081563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:32:14.081563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:14.081563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 644/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:32:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:32:14.087386 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670aa8a-2669-4880-90af-00c799f7b8b4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:14.202919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670aa8a-2669-4880-90af-00c799f7b8b4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:14.241769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a1f35-d8df-42ea-94b3-98e3c60b97f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:14.242462 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a1f35-d8df-42ea-94b3-98e3c60b97f7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:14.242782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a1f35-d8df-42ea-94b3-98e3c60b97f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:14.243096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a1f35-d8df-42ea-94b3-98e3c60b97f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:14.243995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a1f35-d8df-42ea-94b3-98e3c60b97f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:14.244384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a1f35-d8df-42ea-94b3-98e3c60b97f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:14.245828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a1f35-d8df-42ea-94b3-98e3c60b97f7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:14.258960 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 615/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:32:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:14.266536 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d43818-b097-4a94-9a54-caffbf1e24fc None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:14.294370 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:14.377147 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d43818-b097-4a94-9a54-caffbf1e24fc None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:14.417551 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b916105-7f16-4a62-9865-99a2354d91b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:14.418355 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b916105-7f16-4a62-9865-99a2354d91b3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:14.418725 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b916105-7f16-4a62-9865-99a2354d91b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:14.419067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b916105-7f16-4a62-9865-99a2354d91b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:14.420243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b916105-7f16-4a62-9865-99a2354d91b3 None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=f1308deea1ea4629a1ac22d4b903ad4f) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:14.420681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b916105-7f16-4a62-9865-99a2354d91b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:14.422769 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b916105-7f16-4a62-9865-99a2354d91b3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:14.446010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 645/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:32:14 2022] GET /identity/v3/role_assignments?user.id=f1308deea1ea4629a1ac22d4b903ad4f&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=864a6966630544e8bd68bcafba905c6f => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:16.463811 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522aa2fc-eb9e-4051-8df1-9cb0386460a0 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:16.464348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522aa2fc-eb9e-4051-8df1-9cb0386460a0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:16.464348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522aa2fc-eb9e-4051-8df1-9cb0386460a0 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:16.465736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 616/1261] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:32:16.470988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dc91d-3945-4e6e-a484-8b2ed9f1dc95 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:16.471415 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dc91d-3945-4e6e-a484-8b2ed9f1dc95 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:16.471550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dc91d-3945-4e6e-a484-8b2ed9f1dc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:16.487923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5f9dc91d-3945-4e6e-a484-8b2ed9f1dc95 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:32:16.489608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5f9dc91d-3945-4e6e-a484-8b2ed9f1dc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:16.559459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 646/1262] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:32:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:32:16.588145 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522aa2fc-eb9e-4051-8df1-9cb0386460a0 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:16.599598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:16.690209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522aa2fc-eb9e-4051-8df1-9cb0386460a0 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:16.723428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e981cd-a943-41c9-abcd-90ef60cae2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:16.724136 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e981cd-a943-41c9-abcd-90ef60cae2ea None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:16.724416 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e981cd-a943-41c9-abcd-90ef60cae2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:16.724685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e981cd-a943-41c9-abcd-90ef60cae2ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:16.734495 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e981cd-a943-41c9-abcd-90ef60cae2ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:32:16.742787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-b4e981cd-a943-41c9-abcd-90ef60cae2ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:32:16.742787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:16.742787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:16.742787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:16.742787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:16.742787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:16.742787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:16.742787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:16.742787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:16.742787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:16.742787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:16.742787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:16.742787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:32:16.742787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:32:16.742787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:32:16.742787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:32:16.743871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:32:16.743871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:32:16.743871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:32:16.743871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:32:16.743871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:16.743871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:16.743871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:16.743871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:16.743871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:16.743871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:16.743871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:16.743871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:16.743871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:16.743871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:16.743871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:16.743871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:16.744925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:16.744925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:16.744925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:16.744925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:16.744925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:32:16.744925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:32:16.744925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:32:16.744925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:32:16.744925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:32:16.744925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:32:16.744925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:32:16.744925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:32:16.744925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 617/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:32:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:16.750356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9dc91d-3945-4e6e-a484-8b2ed9f1dc95 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:16.768250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:16.848809 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9dc91d-3945-4e6e-a484-8b2ed9f1dc95 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:16.882774 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c33ac-0516-4228-8568-568e729a6cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:16.883432 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c33ac-0516-4228-8568-568e729a6cec None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:16.883717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c33ac-0516-4228-8568-568e729a6cec None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:16.883992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c33ac-0516-4228-8568-568e729a6cec None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:16.884730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c33ac-0516-4228-8568-568e729a6cec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:16.885035 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c33ac-0516-4228-8568-568e729a6cec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:16.886447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c33ac-0516-4228-8568-568e729a6cec None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:16.894482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 647/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:32:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:16.900562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e981cd-a943-41c9-abcd-90ef60cae2ea None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:16.965366 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:16.998911 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e981cd-a943-41c9-abcd-90ef60cae2ea None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:17.031810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ed03f-1732-4f66-b5c8-984d4b177acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:17.032581 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ed03f-1732-4f66-b5c8-984d4b177acd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:17.032865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ed03f-1732-4f66-b5c8-984d4b177acd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:17.033133 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ed03f-1732-4f66-b5c8-984d4b177acd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:17.044054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ed03f-1732-4f66-b5c8-984d4b177acd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:17.045475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ed03f-1732-4f66-b5c8-984d4b177acd None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:17.061874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-523ed03f-1732-4f66-b5c8-984d4b177acd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:17.061874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:17.061874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:17.061874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:17.061874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:17.061874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:17.061874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:17.061874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:17.061874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:17.061874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:17.061874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:17.061874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:17.061874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:32:17.061874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:32:17.061874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:32:17.063037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:32:17.063037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:17.063037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:17.063037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:17.063037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:17.063037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:17.063037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:17.063037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:17.063037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:17.063037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:17.063037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:17.063037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:17.063037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:17.063037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:17.063037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:17.063037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:17.063037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:17.064140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:32:17.064140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:32:17.064140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:17.064140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:32:17.064140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 618/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:32:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:17.068837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c33ac-0516-4228-8568-568e729a6cec None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:17.149128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:17.168590 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c33ac-0516-4228-8568-568e729a6cec None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:17.200652 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc7d76-e0b2-4153-9fbc-0e9f2658f0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:17.201304 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc7d76-e0b2-4153-9fbc-0e9f2658f0dd None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:17.201590 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc7d76-e0b2-4153-9fbc-0e9f2658f0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:17.201868 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc7d76-e0b2-4153-9fbc-0e9f2658f0dd None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:17.202621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc7d76-e0b2-4153-9fbc-0e9f2658f0dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:17.202933 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc7d76-e0b2-4153-9fbc-0e9f2658f0dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:17.204308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc7d76-e0b2-4153-9fbc-0e9f2658f0dd None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:17.216231 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 648/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:17.222040 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ed03f-1732-4f66-b5c8-984d4b177acd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:17.318061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ed03f-1732-4f66-b5c8-984d4b177acd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:17.336978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:17.350546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a5a6f-6d0a-4021-aef7-2c60f11e86f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:17.351335 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a5a6f-6d0a-4021-aef7-2c60f11e86f9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:17.351614 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a5a6f-6d0a-4021-aef7-2c60f11e86f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:17.351883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a5a6f-6d0a-4021-aef7-2c60f11e86f9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:17.359243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a5a6f-6d0a-4021-aef7-2c60f11e86f9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:17.360664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a5a6f-6d0a-4021-aef7-2c60f11e86f9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:17.374756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-266a5a6f-6d0a-4021-aef7-2c60f11e86f9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:32:17.374756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:17.374756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:17.374756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:17.374756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:17.374756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:17.374756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:17.374756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:17.374756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:17.374756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:17.374756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:17.374756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:17.374756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:32:17.374756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:32:17.374756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:32:17.374756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:17.375929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:17.377092 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:17.377092 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:17.377092 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:32:17.377092 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:32:17.377092 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:32:17.377092 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:32:17.377092 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:32:17.377092 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:32:17.377092 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:32:17.377092 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:32:17.377092 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 619/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:32:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:17.382432 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc7d76-e0b2-4153-9fbc-0e9f2658f0dd None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:17.479365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc7d76-e0b2-4153-9fbc-0e9f2658f0dd None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:17.512407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9cf44-1edd-4e35-a9c8-bf71aa5e89ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:17.513067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9cf44-1edd-4e35-a9c8-bf71aa5e89ce None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:17.513349 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9cf44-1edd-4e35-a9c8-bf71aa5e89ce None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:17.513669 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9cf44-1edd-4e35-a9c8-bf71aa5e89ce None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:17.514686 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9cf44-1edd-4e35-a9c8-bf71aa5e89ce None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:17.515230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9cf44-1edd-4e35-a9c8-bf71aa5e89ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:17.516566 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9cf44-1edd-4e35-a9c8-bf71aa5e89ce None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:17.527231 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:17.538144 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 649/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Jul 12 13:32:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:17.544151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a5a6f-6d0a-4021-aef7-2c60f11e86f9 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:17.653647 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a5a6f-6d0a-4021-aef7-2c60f11e86f9 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:17.686319 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c0582-15e0-42ec-8717-6378ad673c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:17.687116 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c0582-15e0-42ec-8717-6378ad673c7b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:17.687397 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c0582-15e0-42ec-8717-6378ad673c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:17.687669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c0582-15e0-42ec-8717-6378ad673c7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:17.698634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c0582-15e0-42ec-8717-6378ad673c7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:17.700078 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c0582-15e0-42ec-8717-6378ad673c7b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:17.716714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-153c0582-15e0-42ec-8717-6378ad673c7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:17.716714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:17.716714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:17.716714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:17.716714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:17.716714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:17.716714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:17.716714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:17.716714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:17.716714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:17.716714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:17.716714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:17.716714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:32:17.716714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:32:17.716714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:32:17.717799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:32:17.717799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:17.717799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:17.717799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:17.717799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:17.717799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:17.717799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:17.717799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:17.717799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:17.717799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:17.717799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:17.717799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:17.717799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:17.717799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:17.717799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:17.717799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:17.717799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:17.719183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:32:17.719183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:32:17.719183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:17.719183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:32:17.719183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 620/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:32:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:17.723660 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9cf44-1edd-4e35-a9c8-bf71aa5e89ce None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:17.820537 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9cf44-1edd-4e35-a9c8-bf71aa5e89ce None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:17.853252 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfcbfd-f812-4d7a-9aa4-76d9be63830d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:17.853920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfcbfd-f812-4d7a-9aa4-76d9be63830d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:17.854238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfcbfd-f812-4d7a-9aa4-76d9be63830d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:17.854515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfcbfd-f812-4d7a-9aa4-76d9be63830d None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:17.855234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dfcbfd-f812-4d7a-9aa4-76d9be63830d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:17.855545 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dfcbfd-f812-4d7a-9aa4-76d9be63830d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:17.856920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dfcbfd-f812-4d7a-9aa4-76d9be63830d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:17.868675 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 650/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:17.874314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c0582-15e0-42ec-8717-6378ad673c7b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:17.905546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:17.972158 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c0582-15e0-42ec-8717-6378ad673c7b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:18.005423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc20f6-4e5b-44d4-83bd-5c0efedda302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:18.006231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc20f6-4e5b-44d4-83bd-5c0efedda302 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:18.006518 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc20f6-4e5b-44d4-83bd-5c0efedda302 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:18.006795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc20f6-4e5b-44d4-83bd-5c0efedda302 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:18.013347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc20f6-4e5b-44d4-83bd-5c0efedda302 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:18.014812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc20f6-4e5b-44d4-83bd-5c0efedda302 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:18.028789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-79bc20f6-4e5b-44d4-83bd-5c0efedda302 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:32:18.028789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:18.028789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:18.028789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:18.028789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:18.028789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:18.028789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:18.028789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:18.028789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:18.028789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:18.028789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:18.028789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:18.028789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:32:18.028789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:32:18.028789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:32:18.029853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:32:18.029853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:18.029853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:18.029853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:18.029853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:18.029853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:18.029853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:18.029853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:18.029853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:18.029853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:18.029853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:18.029853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:18.029853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:18.029853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:18.029853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:18.029853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:18.029853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:18.030854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:32:18.030854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:32:18.030854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:32:18.030854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:32:18.030854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:32:18.030854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:32:18.030854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:32:18.030854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:32:18.030854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 621/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:32:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:18.036458 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfcbfd-f812-4d7a-9aa4-76d9be63830d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:18.091956 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:18.133080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfcbfd-f812-4d7a-9aa4-76d9be63830d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:18.165211 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9318661-13cc-4c2c-a470-add4f5f0714f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:18.165791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9318661-13cc-4c2c-a470-add4f5f0714f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:18.166104 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9318661-13cc-4c2c-a470-add4f5f0714f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:18.166381 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9318661-13cc-4c2c-a470-add4f5f0714f None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:18.167149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9318661-13cc-4c2c-a470-add4f5f0714f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:18.167455 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9318661-13cc-4c2c-a470-add4f5f0714f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:18.168874 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9318661-13cc-4c2c-a470-add4f5f0714f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:18.181616 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 651/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:32:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:18.187593 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc20f6-4e5b-44d4-83bd-5c0efedda302 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:18.276626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:18.286310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc20f6-4e5b-44d4-83bd-5c0efedda302 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:18.322552 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea675f7-3e29-4762-9d1f-b785f06f4ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:18.323432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea675f7-3e29-4762-9d1f-b785f06f4ea2 None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:18.323711 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea675f7-3e29-4762-9d1f-b785f06f4ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:18.323983 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea675f7-3e29-4762-9d1f-b785f06f4ea2 None admin] PATH_INFO: `/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/f1308deea1ea4629a1ac22d4b903ad4f/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:18.325930 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea675f7-3e29-4762-9d1f-b785f06f4ea2 None admin] RBAC: Authorizing `identity:create_grant(project_id=36aa0326fd0545dface07a9ddc308ccd, user_id=f1308deea1ea4629a1ac22d4b903ad4f, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:18.327390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea675f7-3e29-4762-9d1f-b785f06f4ea2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:18.339449 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 622/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jul 12 13:32:18 2022] PUT /identity/v3/projects/36aa0326fd0545dface07a9ddc308ccd/users/f1308deea1ea4629a1ac22d4b903ad4f/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:32:20.331700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e85f7-7552-431d-bc14-6fd45347cb14 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:20.332297 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e85f7-7552-431d-bc14-6fd45347cb14 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:20.332297 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e85f7-7552-431d-bc14-6fd45347cb14 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:20.334194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 652/1274] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:32:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:32:20.339373 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3eb5b-f313-415d-b2c8-2931b49825f7 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:20.339772 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3eb5b-f313-415d-b2c8-2931b49825f7 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:20.339923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3eb5b-f313-415d-b2c8-2931b49825f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:20.356320 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7dc3eb5b-f313-415d-b2c8-2931b49825f7 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:32:20.357513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7dc3eb5b-f313-415d-b2c8-2931b49825f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:20.422825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 623/1275] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:32:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:32:20.446139 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e85f7-7552-431d-bc14-6fd45347cb14 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:20.548495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e85f7-7552-431d-bc14-6fd45347cb14 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:20.566182 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:20.584025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38e5f0-d24e-40ca-a9a0-25ca91a4b93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:20.584748 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38e5f0-d24e-40ca-a9a0-25ca91a4b93f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:20.585030 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38e5f0-d24e-40ca-a9a0-25ca91a4b93f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:20.585305 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38e5f0-d24e-40ca-a9a0-25ca91a4b93f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:20.595219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38e5f0-d24e-40ca-a9a0-25ca91a4b93f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:32:20.602932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-3b38e5f0-d24e-40ca-a9a0-25ca91a4b93f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:32:20.602932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:20.602932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:20.602932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:20.602932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:20.602932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:20.602932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:20.602932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:20.602932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:20.602932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:20.602932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:20.602932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:20.602932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:32:20.602932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:32:20.602932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:32:20.602932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:32:20.604092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:32:20.604092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:32:20.604092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:32:20.604092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:32:20.604092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:20.604092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:20.604092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:20.604092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:20.604092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:20.604092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:20.604092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:20.604092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:20.604092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:20.604092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:20.604092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:20.604092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:20.605203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:20.605203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:20.605203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:20.605203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:20.605203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:32:20.605203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:32:20.605203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:32:20.605203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:32:20.605203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:32:20.605203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:32:20.605203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:32:20.605203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:20.605203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 653/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jul 12 13:32:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:20.610289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3eb5b-f313-415d-b2c8-2931b49825f7 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:20.697501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:20.716886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3eb5b-f313-415d-b2c8-2931b49825f7 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:20.749911 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f1d7e-fb4d-4e89-b996-e38a73399982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:20.750607 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f1d7e-fb4d-4e89-b996-e38a73399982 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:20.750890 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f1d7e-fb4d-4e89-b996-e38a73399982 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:20.751165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f1d7e-fb4d-4e89-b996-e38a73399982 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:20.751903 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f1d7e-fb4d-4e89-b996-e38a73399982 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:20.752209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f1d7e-fb4d-4e89-b996-e38a73399982 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:20.753577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f1d7e-fb4d-4e89-b996-e38a73399982 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:20.762043 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 624/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jul 12 13:32:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:20.767737 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38e5f0-d24e-40ca-a9a0-25ca91a4b93f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:20.873523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38e5f0-d24e-40ca-a9a0-25ca91a4b93f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:20.914280 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5d52b-ab18-4ccf-9bbf-5d7b5e371922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:20.915070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5d52b-ab18-4ccf-9bbf-5d7b5e371922 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:20.915352 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5d52b-ab18-4ccf-9bbf-5d7b5e371922 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:20.915627 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5d52b-ab18-4ccf-9bbf-5d7b5e371922 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:20.926327 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5d52b-ab18-4ccf-9bbf-5d7b5e371922 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:20.927779 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5d52b-ab18-4ccf-9bbf-5d7b5e371922 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:20.944999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-93c5d52b-ab18-4ccf-9bbf-5d7b5e371922 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:20.944999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:20.944999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:20.944999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:20.944999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:20.944999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:20.944999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:20.944999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:20.944999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:20.944999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:20.944999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:20.944999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:20.944999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:32:20.944999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:32:20.944999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:32:20.946154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:32:20.946154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:20.946154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:20.946154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:20.946154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:20.946154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:20.946154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:20.946154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:20.946154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:20.946154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:20.946154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:20.946154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:20.946154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:20.946154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:20.946154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:20.946154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:20.946154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:20.947413 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:32:20.947413 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:32:20.947413 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:20.947413 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:20.947413 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 654/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:32:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:20.951918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f1d7e-fb4d-4e89-b996-e38a73399982 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:21.049189 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f1d7e-fb4d-4e89-b996-e38a73399982 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:21.082281 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435692a6-9bcd-417c-8e5c-8b4d629a5e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:21.082932 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435692a6-9bcd-417c-8e5c-8b4d629a5e56 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:21.083217 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435692a6-9bcd-417c-8e5c-8b4d629a5e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:21.083495 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435692a6-9bcd-417c-8e5c-8b4d629a5e56 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:21.084212 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435692a6-9bcd-417c-8e5c-8b4d629a5e56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:21.084528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435692a6-9bcd-417c-8e5c-8b4d629a5e56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:21.085928 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435692a6-9bcd-417c-8e5c-8b4d629a5e56 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:21.097410 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 625/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:21.103490 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5d52b-ab18-4ccf-9bbf-5d7b5e371922 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:21.134171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:21.201834 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5d52b-ab18-4ccf-9bbf-5d7b5e371922 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:21.234862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede37ef-bdd4-4567-909e-7f73cf12046a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:21.235657 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede37ef-bdd4-4567-909e-7f73cf12046a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:21.235942 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede37ef-bdd4-4567-909e-7f73cf12046a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:21.236217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede37ef-bdd4-4567-909e-7f73cf12046a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:21.244158 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede37ef-bdd4-4567-909e-7f73cf12046a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:21.245608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede37ef-bdd4-4567-909e-7f73cf12046a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:21.259911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-9ede37ef-bdd4-4567-909e-7f73cf12046a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:32:21.259911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:21.259911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:21.259911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:21.259911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:21.259911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:21.259911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:21.259911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:21.259911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:21.259911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:21.259911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:21.259911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:21.259911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:32:21.259911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:32:21.259911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:32:21.259911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:21.261056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:21.262125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:21.262125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:21.262125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:32:21.262125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:32:21.262125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:32:21.262125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:32:21.262125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:32:21.262125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:32:21.262125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:32:21.262125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:21.262125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 655/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:32:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:21.266881 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435692a6-9bcd-417c-8e5c-8b4d629a5e56 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:21.270621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:21.362098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435692a6-9bcd-417c-8e5c-8b4d629a5e56 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:21.395293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb34ec7-1b7b-4fa4-9e10-a32a7c808951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:21.395949 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb34ec7-1b7b-4fa4-9e10-a32a7c808951 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:21.396228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb34ec7-1b7b-4fa4-9e10-a32a7c808951 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:21.396501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb34ec7-1b7b-4fa4-9e10-a32a7c808951 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:21.397319 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb34ec7-1b7b-4fa4-9e10-a32a7c808951 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:21.397623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb34ec7-1b7b-4fa4-9e10-a32a7c808951 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:21.399067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb34ec7-1b7b-4fa4-9e10-a32a7c808951 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:21.416181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 626/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Jul 12 13:32:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:21.421973 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede37ef-bdd4-4567-909e-7f73cf12046a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:21.510381 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:21.522399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede37ef-bdd4-4567-909e-7f73cf12046a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:21.561994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395aed0-620f-4b97-8e1d-a2fb30881b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:21.562665 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395aed0-620f-4b97-8e1d-a2fb30881b0a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:21.562970 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395aed0-620f-4b97-8e1d-a2fb30881b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:21.563226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395aed0-620f-4b97-8e1d-a2fb30881b0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:21.630986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395aed0-620f-4b97-8e1d-a2fb30881b0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:21.632362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395aed0-620f-4b97-8e1d-a2fb30881b0a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:21.722182 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:21.734275 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 656/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jul 12 13:32:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:32:21.740130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb34ec7-1b7b-4fa4-9e10-a32a7c808951 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:21.802020 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:21.839784 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb34ec7-1b7b-4fa4-9e10-a32a7c808951 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:21.872210 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dd80c-1295-400a-8670-ef1198456bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:21.872992 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dd80c-1295-400a-8670-ef1198456bbd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:21.873272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dd80c-1295-400a-8670-ef1198456bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:21.873541 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dd80c-1295-400a-8670-ef1198456bbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:21.884013 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389dd80c-1295-400a-8670-ef1198456bbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:21.885387 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389dd80c-1295-400a-8670-ef1198456bbd None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:21.902893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-389dd80c-1295-400a-8670-ef1198456bbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:21.902893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:21.902893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:21.902893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:21.902893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:21.902893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:21.902893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:21.902893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:21.902893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:21.902893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:21.902893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:21.902893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:21.902893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:32:21.902893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:32:21.902893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:32:21.903958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:32:21.903958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:21.903958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:21.903958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:21.903958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:21.903958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:21.903958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:21.903958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:21.903958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:21.903958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:21.903958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:21.903958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:21.903958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:21.903958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:21.903958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:21.903958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:21.903958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:21.904915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:32:21.904915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:32:21.904915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:32:21.904915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:32:21.904915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 627/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:32:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:21.909707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395aed0-620f-4b97-8e1d-a2fb30881b0a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:22.020991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395aed0-620f-4b97-8e1d-a2fb30881b0a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:22.053164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e170b4-413d-4a6a-a352-3036133c31cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:22.053806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e170b4-413d-4a6a-a352-3036133c31cc None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:22.054121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e170b4-413d-4a6a-a352-3036133c31cc None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:22.054401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e170b4-413d-4a6a-a352-3036133c31cc None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:22.055113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e170b4-413d-4a6a-a352-3036133c31cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:22.055420 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e170b4-413d-4a6a-a352-3036133c31cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:22.056811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e170b4-413d-4a6a-a352-3036133c31cc None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:22.068127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 657/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jul 12 13:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:22.073770 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dd80c-1295-400a-8670-ef1198456bbd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:22.171714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dd80c-1295-400a-8670-ef1198456bbd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:22.185681 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:22.206098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45901447-1295-4e30-9e2c-42e2c241eccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:22.206880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45901447-1295-4e30-9e2c-42e2c241eccb None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:22.207161 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45901447-1295-4e30-9e2c-42e2c241eccb None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:22.207434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45901447-1295-4e30-9e2c-42e2c241eccb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:22.213894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45901447-1295-4e30-9e2c-42e2c241eccb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:22.215364 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45901447-1295-4e30-9e2c-42e2c241eccb None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:22.228974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-45901447-1295-4e30-9e2c-42e2c241eccb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:32:22.228974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:22.228974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:22.228974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:22.228974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:22.228974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:22.228974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:22.228974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:22.228974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:22.228974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:22.228974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:22.228974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:22.228974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:32:22.228974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:32:22.228974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:32:22.230175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:32:22.230175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:22.230175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:22.230175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:22.230175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:22.230175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:22.230175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:22.230175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:22.230175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:22.230175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:22.230175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:22.230175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:22.230175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:22.230175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:22.230175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:22.230175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:22.230175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:22.231347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:32:22.231347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:32:22.231347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:32:22.231347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:32:22.231347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:32:22.231347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:32:22.231347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:32:22.231347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 12 13:32:22.231347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 628/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jul 12 13:32:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:22.236249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e170b4-413d-4a6a-a352-3036133c31cc None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:22.288349 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:22.339888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e170b4-413d-4a6a-a352-3036133c31cc None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:22.372034 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ef0ba-1528-490a-b0e9-102a9b672840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:22.372675 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ef0ba-1528-490a-b0e9-102a9b672840 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:22.372957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ef0ba-1528-490a-b0e9-102a9b672840 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:22.373230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ef0ba-1528-490a-b0e9-102a9b672840 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:22.374035 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ef0ba-1528-490a-b0e9-102a9b672840 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:22.374591 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ef0ba-1528-490a-b0e9-102a9b672840 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:22.376027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ef0ba-1528-490a-b0e9-102a9b672840 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:22.388192 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 658/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jul 12 13:32:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:22.394256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45901447-1295-4e30-9e2c-42e2c241eccb None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:22.491485 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45901447-1295-4e30-9e2c-42e2c241eccb None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:22.526837 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701b36c-6bc7-4bf7-b66d-86cdaf867329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:22.527656 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701b36c-6bc7-4bf7-b66d-86cdaf867329 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:22.527983 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701b36c-6bc7-4bf7-b66d-86cdaf867329 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:22.528261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701b36c-6bc7-4bf7-b66d-86cdaf867329 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:22.529204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701b36c-6bc7-4bf7-b66d-86cdaf867329 None admin] RBAC: Adding query filter params (role.id=864a6966630544e8bd68bcafba905c6f, scope.project.id=36aa0326fd0545dface07a9ddc308ccd, user.id=f1308deea1ea4629a1ac22d4b903ad4f) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:22.529549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701b36c-6bc7-4bf7-b66d-86cdaf867329 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:22.531073 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701b36c-6bc7-4bf7-b66d-86cdaf867329 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:22.538576 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 629/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jul 12 13:32:22 2022] GET /identity/v3/role_assignments?user.id=f1308deea1ea4629a1ac22d4b903ad4f&scope.project.id=36aa0326fd0545dface07a9ddc308ccd&role.id=864a6966630544e8bd68bcafba905c6f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:24.556650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddb02e-6177-4bae-8fd7-4032ab8903c6 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:24.557175 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddb02e-6177-4bae-8fd7-4032ab8903c6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:24.557175 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddb02e-6177-4bae-8fd7-4032ab8903c6 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:24.558607 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 659/1288] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:32:24.563814 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429c4a0-dbe9-4336-b4a6-6298318daeb1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:24.564194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429c4a0-dbe9-4336-b4a6-6298318daeb1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:24.564366 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429c4a0-dbe9-4336-b4a6-6298318daeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:24.581512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a429c4a0-dbe9-4336-b4a6-6298318daeb1 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:32:24.582783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a429c4a0-dbe9-4336-b4a6-6298318daeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:24.610928 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:24.650861 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 630/1289] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:32:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:32:24.672945 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddb02e-6177-4bae-8fd7-4032ab8903c6 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:24.775059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddb02e-6177-4bae-8fd7-4032ab8903c6 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:24.785884 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:24.809938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2173c-9783-4952-9d14-3faba0e8df16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:24.810704 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2173c-9783-4952-9d14-3faba0e8df16 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:24.810986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2173c-9783-4952-9d14-3faba0e8df16 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:24.811263 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2173c-9783-4952-9d14-3faba0e8df16 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:24.811915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2173c-9783-4952-9d14-3faba0e8df16 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:24.813292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2173c-9783-4952-9d14-3faba0e8df16 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:24.825155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-ecf2173c-9783-4952-9d14-3faba0e8df16 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:32:24.825155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:24.825155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:32:24.825155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:32:24.825155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:32:24.825155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:32:24.825155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:32:24.825155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:32:24.825155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:32:24.825155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:24.825155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:32:24.825155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:24.825155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:24.825155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:24.825155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:24.825155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:24.825155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:24.826408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:24.826408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:24.826408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:24.826408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:24.826408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:24.826408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:24.826408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:32:24.826408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:32:24.826408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:32:24.826408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:32:24.826408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:24.826408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:24.826408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:24.826408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:24.826408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:24.826408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:24.827476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:24.827476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:24.827476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:24.827476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:24.827476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:24.827476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:24.827476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:24.827476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:24.827476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:24.827476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:24.827476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:32:24.827476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:32:24.827476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:32:24.827476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:24.827476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 660/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jul 12 13:32:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:24.832362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429c4a0-dbe9-4336-b4a6-6298318daeb1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:24.927806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429c4a0-dbe9-4336-b4a6-6298318daeb1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:24.960064 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d2ea5f-55e6-44c0-8530-841c626e27ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:24.960717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2ea5f-55e6-44c0-8530-841c626e27ed None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:24.960995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2ea5f-55e6-44c0-8530-841c626e27ed None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:24.961266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2ea5f-55e6-44c0-8530-841c626e27ed None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:24.961984 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d2ea5f-55e6-44c0-8530-841c626e27ed None admin] RBAC: Adding query filter params (name=placement) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:24.962290 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d2ea5f-55e6-44c0-8530-841c626e27ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:24.963648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d2ea5f-55e6-44c0-8530-841c626e27ed None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:24.968371 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 631/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Jul 12 13:32:24 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:24.974651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2173c-9783-4952-9d14-3faba0e8df16 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:25.075208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2173c-9783-4952-9d14-3faba0e8df16 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:25.107358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ecfd4-a26f-4699-a3d7-8d245592b703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:25.108020 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ecfd4-a26f-4699-a3d7-8d245592b703 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:25.108301 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ecfd4-a26f-4699-a3d7-8d245592b703 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:25.108576 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ecfd4-a26f-4699-a3d7-8d245592b703 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:25.109275 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ecfd4-a26f-4699-a3d7-8d245592b703 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:25.109585 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ecfd4-a26f-4699-a3d7-8d245592b703 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:25.111020 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ecfd4-a26f-4699-a3d7-8d245592b703 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:25.115978 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 661/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Jul 12 13:32:24 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:27.140409 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3534c77-0114-41de-b664-458e120777c8 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:27.140954 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3534c77-0114-41de-b664-458e120777c8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:27.140954 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3534c77-0114-41de-b664-458e120777c8 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:27.142368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 632/1293] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:32:27.148128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa8fb0-48fd-42c5-bd74-40bc9b6588c9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:27.148553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa8fb0-48fd-42c5-bd74-40bc9b6588c9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:27.148690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa8fb0-48fd-42c5-bd74-40bc9b6588c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:27.165107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d7aa8fb0-48fd-42c5-bd74-40bc9b6588c9 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:32:27.166333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d7aa8fb0-48fd-42c5-bd74-40bc9b6588c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:27.196182 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:27.233686 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 662/1294] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:32:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:32:27.256595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3534c77-0114-41de-b664-458e120777c8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:27.287119 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:27.364569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3534c77-0114-41de-b664-458e120777c8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:27.398822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fcb00f-2246-4332-a0ce-e25a8de31552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:27.399402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcb00f-2246-4332-a0ce-e25a8de31552 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:27.399680 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcb00f-2246-4332-a0ce-e25a8de31552 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:27.399949 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcb00f-2246-4332-a0ce-e25a8de31552 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:27.400655 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fcb00f-2246-4332-a0ce-e25a8de31552 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:27.402028 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fcb00f-2246-4332-a0ce-e25a8de31552 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:27.414620 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 633/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jul 12 13:32:27 2022] POST /identity/v3/services => generated 246 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:32:29.467331 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae468e-5264-42ae-b2b3-3cfe437d6d43 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:29.467892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae468e-5264-42ae-b2b3-3cfe437d6d43 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:29.467892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae468e-5264-42ae-b2b3-3cfe437d6d43 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:29.469253 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 663/1296] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:32:29.474442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78426b-2b25-4a4f-8ee4-29154026a5f1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:29.474921 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78426b-2b25-4a4f-8ee4-29154026a5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:29.475083 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78426b-2b25-4a4f-8ee4-29154026a5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:29.491481 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-5b78426b-2b25-4a4f-8ee4-29154026a5f1 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:32:29.492686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5b78426b-2b25-4a4f-8ee4-29154026a5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:29.580047 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 634/1297] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:32:29 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:32:29.603013 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae468e-5264-42ae-b2b3-3cfe437d6d43 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:29.703644 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae468e-5264-42ae-b2b3-3cfe437d6d43 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:29.735952 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93a0cf-c2a0-4906-bf96-abb52329d734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:29.736670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93a0cf-c2a0-4906-bf96-abb52329d734 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:29.736955 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93a0cf-c2a0-4906-bf96-abb52329d734 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:29.737230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93a0cf-c2a0-4906-bf96-abb52329d734 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:29.737879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93a0cf-c2a0-4906-bf96-abb52329d734 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:29.739795 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93a0cf-c2a0-4906-bf96-abb52329d734 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:29.751720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-ba93a0cf-c2a0-4906-bf96-abb52329d734 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:32:29.751720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:29.751720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:32:29.751720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:32:29.751720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:32:29.751720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:32:29.751720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:32:29.751720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:32:29.751720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:32:29.751720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:29.751720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:32:29.751720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:29.751720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:29.751720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:29.751720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:29.751720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:29.751720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:29.752913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:29.752913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:29.752913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:29.752913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:29.752913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:29.752913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:29.752913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:32:29.752913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:32:29.752913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:32:29.752913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:32:29.752913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:29.752913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:29.752913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:29.752913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:29.752913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:29.752913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:29.754131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:29.754131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:29.754131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:29.754131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:29.754131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:29.754131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:29.754131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:29.754131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:29.754131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:29.754131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:29.754131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:32:29.754131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:32:29.754131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:32:29.754131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:29.754131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 664/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jul 12 13:32:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:29.759544 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78426b-2b25-4a4f-8ee4-29154026a5f1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:29.777338 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:29.855015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78426b-2b25-4a4f-8ee4-29154026a5f1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:29.888358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ec0b1-0945-4426-a8a8-03f5204c7533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:29.889009 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ec0b1-0945-4426-a8a8-03f5204c7533 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:29.889288 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ec0b1-0945-4426-a8a8-03f5204c7533 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:29.889556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ec0b1-0945-4426-a8a8-03f5204c7533 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:29.890279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ec0b1-0945-4426-a8a8-03f5204c7533 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:29.890581 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ec0b1-0945-4426-a8a8-03f5204c7533 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:29.891925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ec0b1-0945-4426-a8a8-03f5204c7533 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:29.896908 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 635/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Jul 12 13:32:29 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:29.902816 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93a0cf-c2a0-4906-bf96-abb52329d734 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:29.933924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:30.000164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93a0cf-c2a0-4906-bf96-abb52329d734 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:30.032506 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28336efa-1e75-44f5-961c-8ccfb70fdcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:30.033151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28336efa-1e75-44f5-961c-8ccfb70fdcf5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:30.033434 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28336efa-1e75-44f5-961c-8ccfb70fdcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:30.033709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28336efa-1e75-44f5-961c-8ccfb70fdcf5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:30.034514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28336efa-1e75-44f5-961c-8ccfb70fdcf5 None admin] RBAC: Adding query filter params (interface=public, service_id=e218567ab7ff498db611152347db0e64, region_id=RegionOne) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:30.034827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28336efa-1e75-44f5-961c-8ccfb70fdcf5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:30.036703 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28336efa-1e75-44f5-961c-8ccfb70fdcf5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:30.041604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 665/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Jul 12 13:32:29 2022] GET /identity/v3/endpoints?service_id=e218567ab7ff498db611152347db0e64&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:30.047171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ec0b1-0945-4426-a8a8-03f5204c7533 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:30.125071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:30.142370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ec0b1-0945-4426-a8a8-03f5204c7533 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:30.174816 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f0817-770c-455a-9151-88170f3b3568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:30.175468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f0817-770c-455a-9151-88170f3b3568 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:30.176295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f0817-770c-455a-9151-88170f3b3568 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:30.176569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f0817-770c-455a-9151-88170f3b3568 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:30.177239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f0817-770c-455a-9151-88170f3b3568 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:30.178620 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f0817-770c-455a-9151-88170f3b3568 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:30.185236 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 636/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jul 12 13:32:30 2022] GET /identity/v3/services => generated 2191 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:32:32.157627 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668047bc-ba67-4ae4-a9c9-113329a86502 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:32.158207 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668047bc-ba67-4ae4-a9c9-113329a86502 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:32.158313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668047bc-ba67-4ae4-a9c9-113329a86502 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:32.159594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 666/1302] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:32:32.165282 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f2a8c-f330-491e-b17a-bdd3a0aba952 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:32.165683 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f2a8c-f330-491e-b17a-bdd3a0aba952 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:32.165827 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f2a8c-f330-491e-b17a-bdd3a0aba952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:32.181936 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7b3f2a8c-f330-491e-b17a-bdd3a0aba952 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:32:32.183136 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7b3f2a8c-f330-491e-b17a-bdd3a0aba952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:32.246464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 637/1303] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:32:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:32:32.269209 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668047bc-ba67-4ae4-a9c9-113329a86502 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:32.362521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:32.370080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668047bc-ba67-4ae4-a9c9-113329a86502 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:32.403611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b72f6-db94-4e04-a54b-f2b4b8229809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:32.404403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b72f6-db94-4e04-a54b-f2b4b8229809 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:32.404691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b72f6-db94-4e04-a54b-f2b4b8229809 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:32.404970 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b72f6-db94-4e04-a54b-f2b4b8229809 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:32.405710 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b72f6-db94-4e04-a54b-f2b4b8229809 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:32.407129 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b72f6-db94-4e04-a54b-f2b4b8229809 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:32.418927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-5e7b72f6-db94-4e04-a54b-f2b4b8229809 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:32:32.418927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:32.418927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:32:32.418927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:32:32.418927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:32:32.418927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:32:32.418927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:32:32.418927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:32:32.418927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:32:32.418927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:32.418927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:32:32.418927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:32.418927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:32:32.418927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:32:32.418927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:32:32.418927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:32:32.418927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:32:32.420071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:32:32.420071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:32:32.420071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:32:32.420071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:32:32.420071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:32:32.420071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:32:32.420071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:32:32.420071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:32:32.420071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:32:32.420071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:32:32.420071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:32:32.420071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:32:32.420071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:32:32.420071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:32:32.420071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:32:32.420071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:32:32.421117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:32:32.421117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 12 13:32:32.421117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:32:32.421117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:32:32.421117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:32:32.421117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:32:32.421117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:32:32.421117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:32:32.421117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:32:32.421117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:32:32.421117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:32:32.421117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:32:32.421117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:32:32.421117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 12 13:32:32.421117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 667/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jul 12 13:32:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:32:32.426160 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f2a8c-f330-491e-b17a-bdd3a0aba952 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:32.497765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:32.521100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f2a8c-f330-491e-b17a-bdd3a0aba952 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:32.554006 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999df69-436d-4e47-a9a1-613debec5335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:32.554658 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999df69-436d-4e47-a9a1-613debec5335 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:32.554939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999df69-436d-4e47-a9a1-613debec5335 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:32.555203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999df69-436d-4e47-a9a1-613debec5335 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:32.555893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999df69-436d-4e47-a9a1-613debec5335 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:32:32.556196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999df69-436d-4e47-a9a1-613debec5335 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:32.557554 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999df69-436d-4e47-a9a1-613debec5335 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:32.562638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 638/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Jul 12 13:32:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:32:32.569144 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b72f6-db94-4e04-a54b-f2b4b8229809 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:32.666608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b72f6-db94-4e04-a54b-f2b4b8229809 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:32.700418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a683d0-b4c4-4534-ba35-2a7f61e7e30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:32.700993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a683d0-b4c4-4534-ba35-2a7f61e7e30c None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:32.701281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a683d0-b4c4-4534-ba35-2a7f61e7e30c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:32.701560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a683d0-b4c4-4534-ba35-2a7f61e7e30c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:32.702324 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a683d0-b4c4-4534-ba35-2a7f61e7e30c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:32.703705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a683d0-b4c4-4534-ba35-2a7f61e7e30c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:32.723069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 668/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jul 12 13:32:32 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:32:42.560041 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=30 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:42.718619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:35.538434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dd1b7-307d-4b51-aceb-fdbd267356e2 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:35.538967 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dd1b7-307d-4b51-aceb-fdbd267356e2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:35.538967 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dd1b7-307d-4b51-aceb-fdbd267356e2 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:35.541180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 639/1307] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:33:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:33:35.546710 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b85011-424a-40c8-b487-ca604fdee358 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:35.547870 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b85011-424a-40c8-b487-ca604fdee358 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:35.548422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b85011-424a-40c8-b487-ca604fdee358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:35.702152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-81b85011-424a-40c8-b487-ca604fdee358 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:35.703481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-81b85011-424a-40c8-b487-ca604fdee358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:35.945781 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 669/1308] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:33:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:36.009329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1706a-5693-4562-8166-08195a77675d None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:36.010010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1706a-5693-4562-8166-08195a77675d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:36.010511 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1706a-5693-4562-8166-08195a77675d None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:36.012382 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 640/1309] 10.222.0.52 () {54 vars in 964 bytes} [Tue Jul 12 13:33:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:33:36.039652 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188174a4-de17-4d6c-ab47-21b868104cf1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:36.040383 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188174a4-de17-4d6c-ab47-21b868104cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:36.040904 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188174a4-de17-4d6c-ab47-21b868104cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:36.172947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-188174a4-de17-4d6c-ab47-21b868104cf1 None None] MFA Rules not processed for user `3e833722fe4e493e9ddc4b275632536f`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:36.174639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-188174a4-de17-4d6c-ab47-21b868104cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:36.202921 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:36.283721 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1706a-5693-4562-8166-08195a77675d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:36.286537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c6d1706a-5693-4562-8166-08195a77675d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:36.290827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 670/1310] 10.222.0.52 () {58 vars in 1061 bytes} [Tue Jul 12 13:33:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:36.434709 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1706a-5693-4562-8166-08195a77675d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:36.529512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510e990-9b14-4fa5-af45-79637b6c5ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:36.530256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510e990-9b14-4fa5-af45-79637b6c5ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:36.530540 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510e990-9b14-4fa5-af45-79637b6c5ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:36.530813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510e990-9b14-4fa5-af45-79637b6c5ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:36.532465 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1510e990-9b14-4fa5-af45-79637b6c5ba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:36.596264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:36.668511 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1510e990-9b14-4fa5-af45-79637b6c5ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:36.670281 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1510e990-9b14-4fa5-af45-79637b6c5ba2 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:36.698669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 641/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:33:39.058275 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692f89d-483b-4ec4-93dc-d7294144c8e6 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:39.058805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692f89d-483b-4ec4-93dc-d7294144c8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:39.058899 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692f89d-483b-4ec4-93dc-d7294144c8e6 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:39.060204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 671/1312] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:33:39.066001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb18a4-135f-46fe-9d08-6df31c2d0c79 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:39.066448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb18a4-135f-46fe-9d08-6df31c2d0c79 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:39.066567 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb18a4-135f-46fe-9d08-6df31c2d0c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:39.158359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f3eb18a4-135f-46fe-9d08-6df31c2d0c79 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:39.159617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f3eb18a4-135f-46fe-9d08-6df31c2d0c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:39.261455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 642/1313] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:39.298981 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692f89d-483b-4ec4-93dc-d7294144c8e6 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:39.300609 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2692f89d-483b-4ec4-93dc-d7294144c8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:39.425225 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692f89d-483b-4ec4-93dc-d7294144c8e6 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:39.498798 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:39.501920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8809737-85ae-43f4-b353-a8c900e6c120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:39.502532 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8809737-85ae-43f4-b353-a8c900e6c120 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:39.503793 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8809737-85ae-43f4-b353-a8c900e6c120 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:39.504282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8809737-85ae-43f4-b353-a8c900e6c120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:39.506369 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b8809737-85ae-43f4-b353-a8c900e6c120 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:39.617573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8809737-85ae-43f4-b353-a8c900e6c120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:39.619058 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8809737-85ae-43f4-b353-a8c900e6c120 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:39.638148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 672/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:33:41.968089 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed986324-911b-49df-8d06-9ddad6b732d4 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:41.968646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed986324-911b-49df-8d06-9ddad6b732d4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:41.968646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed986324-911b-49df-8d06-9ddad6b732d4 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:41.970043 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 643/1315] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:33:41.975722 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8899289-ca1a-4cbf-a562-0047f0a75395 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:41.976169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8899289-ca1a-4cbf-a562-0047f0a75395 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:41.976292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8899289-ca1a-4cbf-a562-0047f0a75395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:42.040512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a8899289-ca1a-4cbf-a562-0047f0a75395 None None] MFA Rules not processed for user `febf11689a9345e18b3809a7135a9b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:42.041740 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a8899289-ca1a-4cbf-a562-0047f0a75395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:42.092068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:42.136636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 673/1316] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:33:41 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:42.188395 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cbe3d-2efd-4c96-a397-eeb38b50deea None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:42.188681 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cbe3d-2efd-4c96-a397-eeb38b50deea None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:42.188963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cbe3d-2efd-4c96-a397-eeb38b50deea None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:42.190873 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 644/1317] 10.222.0.52 () {54 vars in 964 bytes} [Tue Jul 12 13:33:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:33:42.196372 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bf821-96e7-408d-9e21-f0b72abe6491 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:42.196667 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bf821-96e7-408d-9e21-f0b72abe6491 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:42.196952 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bf821-96e7-408d-9e21-f0b72abe6491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:42.270559 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1a5bf821-96e7-408d-9e21-f0b72abe6491 None None] MFA Rules not processed for user `3e833722fe4e493e9ddc4b275632536f`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:42.271779 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1a5bf821-96e7-408d-9e21-f0b72abe6491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:42.345633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 674/1318] 10.222.0.52 () {58 vars in 1061 bytes} [Tue Jul 12 13:33:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:42.354939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419cbe3d-2efd-4c96-a397-eeb38b50deea None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:42.356518 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-419cbe3d-2efd-4c96-a397-eeb38b50deea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:42.366987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:42.460659 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419cbe3d-2efd-4c96-a397-eeb38b50deea None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:42.532636 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fc791-247d-4ef6-9c32-e412a3b4b675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:42.533269 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fc791-247d-4ef6-9c32-e412a3b4b675 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:42.533546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fc791-247d-4ef6-9c32-e412a3b4b675 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:42.533814 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fc791-247d-4ef6-9c32-e412a3b4b675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:42.535492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-dd7fc791-247d-4ef6-9c32-e412a3b4b675 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:42.596459 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:42.643651 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fc791-247d-4ef6-9c32-e412a3b4b675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:42.645085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fc791-247d-4ef6-9c32-e412a3b4b675 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:42.663203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 645/1319] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:33:45.598940 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202e891-999e-4f60-967f-b7283ace360c None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:45.599462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202e891-999e-4f60-967f-b7283ace360c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:45.599577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202e891-999e-4f60-967f-b7283ace360c None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:45.600858 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 675/1320] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:33:45.606114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97610e9d-8267-495d-888c-c4d4d6876b8f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:45.606493 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97610e9d-8267-495d-888c-c4d4d6876b8f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:45.606664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97610e9d-8267-495d-888c-c4d4d6876b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:45.670337 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-97610e9d-8267-495d-888c-c4d4d6876b8f None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:45.671599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-97610e9d-8267-495d-888c-c4d4d6876b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:45.766954 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 646/1321] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:33:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:45.806278 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202e891-999e-4f60-967f-b7283ace360c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:45.807822 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1202e891-999e-4f60-967f-b7283ace360c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:45.857569 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:45.912233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202e891-999e-4f60-967f-b7283ace360c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:45.982694 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ee159-53f8-43bd-b072-ff101a7b2581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:45.983335 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ee159-53f8-43bd-b072-ff101a7b2581 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:45.983616 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ee159-53f8-43bd-b072-ff101a7b2581 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:45.983891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ee159-53f8-43bd-b072-ff101a7b2581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:45.985511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-145ee159-53f8-43bd-b072-ff101a7b2581 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:46.082613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:46.096078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ee159-53f8-43bd-b072-ff101a7b2581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:46.098218 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ee159-53f8-43bd-b072-ff101a7b2581 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:46.116372 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 676/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:33:48.833694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6f4bd-6c01-40f3-9474-9fe0aecf281c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:48.834327 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6f4bd-6c01-40f3-9474-9fe0aecf281c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:48.834327 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6f4bd-6c01-40f3-9474-9fe0aecf281c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:48.835670 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 647/1323] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:33:48.841095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585d981-44c2-48a9-a61d-52d85c165b73 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:48.841544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585d981-44c2-48a9-a61d-52d85c165b73 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:48.841657 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585d981-44c2-48a9-a61d-52d85c165b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:48.907523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a585d981-44c2-48a9-a61d-52d85c165b73 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:48.909262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a585d981-44c2-48a9-a61d-52d85c165b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:49.022230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 677/1324] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:33:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:49.059004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6f4bd-6c01-40f3-9474-9fe0aecf281c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:49.060570 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8aa6f4bd-6c01-40f3-9474-9fe0aecf281c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:49.081297 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:49.171227 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6f4bd-6c01-40f3-9474-9fe0aecf281c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:49.250950 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862df01e-22d7-4d0f-88da-53faa695a81a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:49.251586 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862df01e-22d7-4d0f-88da-53faa695a81a service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:49.251865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862df01e-22d7-4d0f-88da-53faa695a81a service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:49.252133 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862df01e-22d7-4d0f-88da-53faa695a81a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:49.253784 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-862df01e-22d7-4d0f-88da-53faa695a81a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:49.331230 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:49.369335 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862df01e-22d7-4d0f-88da-53faa695a81a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:49.370704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862df01e-22d7-4d0f-88da-53faa695a81a service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:49.388700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 648/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:33:51.491530 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bea839-5a24-4033-8e13-79949d46b866 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:51.491996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bea839-5a24-4033-8e13-79949d46b866 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:51.492098 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bea839-5a24-4033-8e13-79949d46b866 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:51.493464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 678/1326] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:33:51.499239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec44a1-8099-45ef-8f0e-c93f568fd8dd None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:51.499718 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec44a1-8099-45ef-8f0e-c93f568fd8dd None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:51.499824 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec44a1-8099-45ef-8f0e-c93f568fd8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:51.565432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-58ec44a1-8099-45ef-8f0e-c93f568fd8dd None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:51.566681 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-58ec44a1-8099-45ef-8f0e-c93f568fd8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:51.665207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 649/1327] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:51.701632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bea839-5a24-4033-8e13-79949d46b866 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:51.703314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-30bea839-5a24-4033-8e13-79949d46b866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:51.766158 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:51.811673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bea839-5a24-4033-8e13-79949d46b866 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:51.896751 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d8c6a-64e3-4fc3-9706-aff1fcdbb634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:51.897404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d8c6a-64e3-4fc3-9706-aff1fcdbb634 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:51.897688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d8c6a-64e3-4fc3-9706-aff1fcdbb634 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:51.897981 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d8c6a-64e3-4fc3-9706-aff1fcdbb634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:51.899648 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1f8d8c6a-64e3-4fc3-9706-aff1fcdbb634 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:52.005560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:52.008168 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d8c6a-64e3-4fc3-9706-aff1fcdbb634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:52.010190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d8c6a-64e3-4fc3-9706-aff1fcdbb634 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:52.035671 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 679/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:33:54.175420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c4de5-fff6-4613-9b90-561c1aa5466b None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:54.175931 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c4de5-fff6-4613-9b90-561c1aa5466b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:54.176040 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c4de5-fff6-4613-9b90-561c1aa5466b None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:54.177348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 650/1329] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:33:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:33:54.182971 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5f32a-4d79-45a8-bd84-b3eeced2b4e0 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:54.183465 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5f32a-4d79-45a8-bd84-b3eeced2b4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:54.183582 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5f32a-4d79-45a8-bd84-b3eeced2b4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:54.257373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a8f5f32a-4d79-45a8-bd84-b3eeced2b4e0 None None] MFA Rules not processed for user `febf11689a9345e18b3809a7135a9b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:54.258671 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a8f5f32a-4d79-45a8-bd84-b3eeced2b4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:54.357350 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 680/1330] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:33:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:54.395251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c4de5-fff6-4613-9b90-561c1aa5466b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:54.396852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e30c4de5-fff6-4613-9b90-561c1aa5466b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:54.428528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:54.503433 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c4de5-fff6-4613-9b90-561c1aa5466b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:54.575247 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d45d90-74c4-46b7-bf16-95ab3efde219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:54.575893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d45d90-74c4-46b7-bf16-95ab3efde219 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:54.576179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d45d90-74c4-46b7-bf16-95ab3efde219 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:54.576457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d45d90-74c4-46b7-bf16-95ab3efde219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:54.578139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-13d45d90-74c4-46b7-bf16-95ab3efde219 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:54.686723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:54.721339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d45d90-74c4-46b7-bf16-95ab3efde219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:54.723337 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d45d90-74c4-46b7-bf16-95ab3efde219 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:54.742796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 651/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:33:57.021072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8c0eb-8033-4277-bc94-7ca1aef86be8 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:57.021608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8c0eb-8033-4277-bc94-7ca1aef86be8 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:57.021726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8c0eb-8033-4277-bc94-7ca1aef86be8 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:57.023075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 681/1332] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:33:57.028197 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9803a-b2f7-4397-a0e0-2fcd5a139355 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:57.028596 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9803a-b2f7-4397-a0e0-2fcd5a139355 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:57.028750 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9803a-b2f7-4397-a0e0-2fcd5a139355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:57.091564 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-38f9803a-b2f7-4397-a0e0-2fcd5a139355 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:57.092767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-38f9803a-b2f7-4397-a0e0-2fcd5a139355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:57.191434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 652/1333] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:57.226378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8c0eb-8033-4277-bc94-7ca1aef86be8 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:57.227906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9ef8c0eb-8033-4277-bc94-7ca1aef86be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:57.297789 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:57.333435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8c0eb-8033-4277-bc94-7ca1aef86be8 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:57.405859 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480285bc-8ae2-4648-abc8-942fb0900f4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:57.406533 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480285bc-8ae2-4648-abc8-942fb0900f4d service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:57.406816 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480285bc-8ae2-4648-abc8-942fb0900f4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:57.407088 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480285bc-8ae2-4648-abc8-942fb0900f4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:57.409316 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-480285bc-8ae2-4648-abc8-942fb0900f4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:57.518676 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480285bc-8ae2-4648-abc8-942fb0900f4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:57.520142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480285bc-8ae2-4648-abc8-942fb0900f4d service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:57.531001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:57.540901 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 682/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:33:59.608170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc5f69-58ba-4cc1-8e6c-71b77a33f5ee None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:59.608722 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc5f69-58ba-4cc1-8e6c-71b77a33f5ee None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:59.608722 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc5f69-58ba-4cc1-8e6c-71b77a33f5ee None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:59.610097 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 653/1335] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:33:59.615726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b0c91-aa4a-4501-a854-b7109937bed2 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:59.616117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b0c91-aa4a-4501-a854-b7109937bed2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:59.616325 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b0c91-aa4a-4501-a854-b7109937bed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:59.681939 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c46b0c91-aa4a-4501-a854-b7109937bed2 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:59.683200 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c46b0c91-aa4a-4501-a854-b7109937bed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:59.780089 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 683/1336] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:33:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:59.819918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc5f69-58ba-4cc1-8e6c-71b77a33f5ee None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:59.821453 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-82bc5f69-58ba-4cc1-8e6c-71b77a33f5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:59.862804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:59.933441 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc5f69-58ba-4cc1-8e6c-71b77a33f5ee None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:00.004974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf28f22-efc0-4c64-ad2c-79a9838cddcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:00.005610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf28f22-efc0-4c64-ad2c-79a9838cddcc service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:00.005892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf28f22-efc0-4c64-ad2c-79a9838cddcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:00.006196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf28f22-efc0-4c64-ad2c-79a9838cddcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:00.007837 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7bf28f22-efc0-4c64-ad2c-79a9838cddcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:00.088425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:00.115048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf28f22-efc0-4c64-ad2c-79a9838cddcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:00.116505 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf28f22-efc0-4c64-ad2c-79a9838cddcc service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:00.134470 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 654/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:34:02.760653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45053f32-3707-4389-986b-1b07d1e3b19a None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:02.761180 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45053f32-3707-4389-986b-1b07d1e3b19a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:02.761180 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45053f32-3707-4389-986b-1b07d1e3b19a None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:02.762511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 684/1338] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:02.768255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad99c83-110c-4dcc-b1de-a36f5774da2e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:02.769242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad99c83-110c-4dcc-b1de-a36f5774da2e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:02.769242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad99c83-110c-4dcc-b1de-a36f5774da2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:02.833973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-bad99c83-110c-4dcc-b1de-a36f5774da2e None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:02.835220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bad99c83-110c-4dcc-b1de-a36f5774da2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:02.930456 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 655/1339] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:34:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:34:02.965853 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45053f32-3707-4389-986b-1b07d1e3b19a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:02.967440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-45053f32-3707-4389-986b-1b07d1e3b19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:03.042856 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:03.071438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45053f32-3707-4389-986b-1b07d1e3b19a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:03.143238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d852aa-87c6-4df4-90f3-547dac6766bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:03.143875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d852aa-87c6-4df4-90f3-547dac6766bc service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:03.144155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d852aa-87c6-4df4-90f3-547dac6766bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:03.144425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d852aa-87c6-4df4-90f3-547dac6766bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:03.146080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-17d852aa-87c6-4df4-90f3-547dac6766bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:03.253081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d852aa-87c6-4df4-90f3-547dac6766bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:03.254607 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d852aa-87c6-4df4-90f3-547dac6766bc service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:03.274779 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 685/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:34:07.423742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c320c-8687-4ab5-ba2f-f19a6e19248d None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:07.424278 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c320c-8687-4ab5-ba2f-f19a6e19248d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:07.424278 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c320c-8687-4ab5-ba2f-f19a6e19248d None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:07.425667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 656/1341] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:07.431075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5faa86-47a1-4c27-b621-b8b20f7ed27a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:07.431491 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5faa86-47a1-4c27-b621-b8b20f7ed27a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:07.431608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5faa86-47a1-4c27-b621-b8b20f7ed27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:07.435866 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:07.497425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7a5faa86-47a1-4c27-b621-b8b20f7ed27a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:07.498691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7a5faa86-47a1-4c27-b621-b8b20f7ed27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:07.600268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 686/1342] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:07.639778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c320c-8687-4ab5-ba2f-f19a6e19248d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:07.641479 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-961c320c-8687-4ab5-ba2f-f19a6e19248d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:07.691752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:07.750516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c320c-8687-4ab5-ba2f-f19a6e19248d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:07.826839 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baa0df-85a7-4b9c-acee-25b3ba41dc7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:07.827489 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baa0df-85a7-4b9c-acee-25b3ba41dc7e service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:07.827774 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baa0df-85a7-4b9c-acee-25b3ba41dc7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:07.828051 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baa0df-85a7-4b9c-acee-25b3ba41dc7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:07.829690 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b4baa0df-85a7-4b9c-acee-25b3ba41dc7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:07.932359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:07.950031 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4baa0df-85a7-4b9c-acee-25b3ba41dc7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:07.951406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4baa0df-85a7-4b9c-acee-25b3ba41dc7e service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:07.971723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 657/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:34:10.652546 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea35763-1523-4bdb-8e0e-b1100d98ea9e None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:10.653128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea35763-1523-4bdb-8e0e-b1100d98ea9e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:10.653128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea35763-1523-4bdb-8e0e-b1100d98ea9e None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:10.654699 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 687/1344] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:10.660792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e6b6e-6c3b-4437-b4d8-881f253dacf3 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:10.661408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e6b6e-6c3b-4437-b4d8-881f253dacf3 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:10.661408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e6b6e-6c3b-4437-b4d8-881f253dacf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:10.740241 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9f2e6b6e-6c3b-4437-b4d8-881f253dacf3 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:10.741499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9f2e6b6e-6c3b-4437-b4d8-881f253dacf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:10.856330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 658/1345] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:34:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:34:10.895005 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea35763-1523-4bdb-8e0e-b1100d98ea9e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:10.896529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cea35763-1523-4bdb-8e0e-b1100d98ea9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:10.984305 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:11.002998 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea35763-1523-4bdb-8e0e-b1100d98ea9e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:11.077911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123a772-456c-4853-93fd-386cf03862b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:11.078633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123a772-456c-4853-93fd-386cf03862b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:11.078915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123a772-456c-4853-93fd-386cf03862b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:11.079188 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123a772-456c-4853-93fd-386cf03862b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:11.081554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3123a772-456c-4853-93fd-386cf03862b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:11.190953 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123a772-456c-4853-93fd-386cf03862b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:11.192405 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123a772-456c-4853-93fd-386cf03862b5 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:11.212210 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 688/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:34:15.541530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b796170-14bf-43a8-bd77-b34be2a95de5 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:15.542095 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b796170-14bf-43a8-bd77-b34be2a95de5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:15.542245 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b796170-14bf-43a8-bd77-b34be2a95de5 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:15.543694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 659/1347] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:15.549740 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003d9ef-dc60-4e72-96d6-a9b1f34c8fc0 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:15.550174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003d9ef-dc60-4e72-96d6-a9b1f34c8fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:15.550335 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003d9ef-dc60-4e72-96d6-a9b1f34c8fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:15.566295 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:15.632332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9003d9ef-dc60-4e72-96d6-a9b1f34c8fc0 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:15.633492 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9003d9ef-dc60-4e72-96d6-a9b1f34c8fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:15.745636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 689/1348] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:34:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:34:15.784285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b796170-14bf-43a8-bd77-b34be2a95de5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:15.785844 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8b796170-14bf-43a8-bd77-b34be2a95de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:15.848368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:15.897051 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b796170-14bf-43a8-bd77-b34be2a95de5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:15.973572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e91e73-4b05-4a7f-abcb-94ca4afa2459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:15.974266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e91e73-4b05-4a7f-abcb-94ca4afa2459 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:15.974549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e91e73-4b05-4a7f-abcb-94ca4afa2459 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:15.974823 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e91e73-4b05-4a7f-abcb-94ca4afa2459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:15.976474 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-45e91e73-4b05-4a7f-abcb-94ca4afa2459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:16.090731 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:16.094922 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e91e73-4b05-4a7f-abcb-94ca4afa2459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:16.096950 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e91e73-4b05-4a7f-abcb-94ca4afa2459 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:16.116011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 660/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:34:20.996346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3977b92-458f-46af-88dc-dc3351b30e05 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:21.009280 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3977b92-458f-46af-88dc-dc3351b30e05 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:21.009585 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3977b92-458f-46af-88dc-dc3351b30e05 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:21.011348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 690/1350] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jul 12 13:34:20 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:21.017866 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552eef1-744c-40b1-92a6-97af998ef5ca None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:21.018402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552eef1-744c-40b1-92a6-97af998ef5ca None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:21.018607 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552eef1-744c-40b1-92a6-97af998ef5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:21.120752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9552eef1-744c-40b1-92a6-97af998ef5ca None None] MFA Rules not processed for user `f1308deea1ea4629a1ac22d4b903ad4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:21.122285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9552eef1-744c-40b1-92a6-97af998ef5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:21.263185 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 661/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jul 12 13:34:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:25.266593 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6f76a-ee36-4d8d-a28d-8960e9315b0f None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:25.267195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6f76a-ee36-4d8d-a28d-8960e9315b0f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:25.267195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6f76a-ee36-4d8d-a28d-8960e9315b0f None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:25.268747 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 691/1352] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jul 12 13:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:25.273505 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca566083-3b41-4141-ac7e-c286b4e1f60c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:25.273949 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca566083-3b41-4141-ac7e-c286b4e1f60c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:25.274077 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca566083-3b41-4141-ac7e-c286b4e1f60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:25.335031 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:25.359604 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ca566083-3b41-4141-ac7e-c286b4e1f60c None None] MFA Rules not processed for user `f1308deea1ea4629a1ac22d4b903ad4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:25.361365 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ca566083-3b41-4141-ac7e-c286b4e1f60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:25.484103 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 662/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jul 12 13:34:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:25.743522 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=59 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:27.013230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa76c8-3876-4a61-8af2-646da3ab818a None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:27.013862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa76c8-3876-4a61-8af2-646da3ab818a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:27.013862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa76c8-3876-4a61-8af2-646da3ab818a None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:27.015214 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 692/1354] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jul 12 13:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:27.021054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27481e59-8640-4a9d-be13-0c07e60db60e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:27.021485 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27481e59-8640-4a9d-be13-0c07e60db60e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:27.021627 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27481e59-8640-4a9d-be13-0c07e60db60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:27.097503 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-27481e59-8640-4a9d-be13-0c07e60db60e None None] MFA Rules not processed for user `f1308deea1ea4629a1ac22d4b903ad4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:27.098678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-27481e59-8640-4a9d-be13-0c07e60db60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:27.144044 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:27.210599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 663/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jul 12 13:34:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:30.760113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4e83d-0854-4811-ab87-33254f4e9246 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:30.760611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4e83d-0854-4811-ab87-33254f4e9246 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:30.760611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4e83d-0854-4811-ab87-33254f4e9246 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:30.766991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 693/1356] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:34:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:30.768452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd25d5b-0c4d-4127-afde-4fac4c91974e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:30.768914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd25d5b-0c4d-4127-afde-4fac4c91974e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:30.769093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd25d5b-0c4d-4127-afde-4fac4c91974e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:30.851722 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-dfd25d5b-0c4d-4127-afde-4fac4c91974e None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:30.853217 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-dfd25d5b-0c4d-4127-afde-4fac4c91974e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:30.961734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 664/1357] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:31.019714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d8b75-b436-4597-abe4-7b1e0098b30c None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:31.020026 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d8b75-b436-4597-abe4-7b1e0098b30c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:31.021169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d8b75-b436-4597-abe4-7b1e0098b30c None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:31.022775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 694/1358] 10.222.0.52 () {54 vars in 961 bytes} [Tue Jul 12 13:34:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:31.044385 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dfe60-d20c-4777-9215-f6d861463e76 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:31.044700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dfe60-d20c-4777-9215-f6d861463e76 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:31.045365 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dfe60-d20c-4777-9215-f6d861463e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:31.068489 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:31.142060 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a98dfe60-d20c-4777-9215-f6d861463e76 None None] MFA Rules not processed for user `0d3915cde0d04bcfb9b5ce8c89af1e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:31.143385 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a98dfe60-d20c-4777-9215-f6d861463e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:31.277851 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 665/1359] 10.222.0.52 () {58 vars in 1058 bytes} [Tue Jul 12 13:34:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:31.290672 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d8b75-b436-4597-abe4-7b1e0098b30c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:31.292429 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f08d8b75-b436-4597-abe4-7b1e0098b30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:31.457041 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d8b75-b436-4597-abe4-7b1e0098b30c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:31.555975 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:31.601072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a023d-8c23-4f07-b45b-beaed6b219b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:31.601730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a023d-8c23-4f07-b45b-beaed6b219b1 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:31.602040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a023d-8c23-4f07-b45b-beaed6b219b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:31.602321 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a023d-8c23-4f07-b45b-beaed6b219b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:31.603952 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-240a023d-8c23-4f07-b45b-beaed6b219b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:31.710112 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a023d-8c23-4f07-b45b-beaed6b219b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:31.711907 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a023d-8c23-4f07-b45b-beaed6b219b1 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:31.731660 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 695/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:34:33.988279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290242d3-4b06-4afa-96ce-630dfcba9d4c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:33.989032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290242d3-4b06-4afa-96ce-630dfcba9d4c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:33.989032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290242d3-4b06-4afa-96ce-630dfcba9d4c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:33.990150 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 666/1361] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:33.995784 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b36ba9-0ec0-411f-a3ae-c30dff2df16a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:33.996236 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b36ba9-0ec0-411f-a3ae-c30dff2df16a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:33.996465 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b36ba9-0ec0-411f-a3ae-c30dff2df16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:34.079162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b6b36ba9-0ec0-411f-a3ae-c30dff2df16a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:34.081174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b6b36ba9-0ec0-411f-a3ae-c30dff2df16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:34.122392 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:34.200250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 696/1362] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:34.247087 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290242d3-4b06-4afa-96ce-630dfcba9d4c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:34.248530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-290242d3-4b06-4afa-96ce-630dfcba9d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:34.320850 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:34.421251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290242d3-4b06-4afa-96ce-630dfcba9d4c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:34.553334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15634924-3e3a-4f09-b30c-7d469832116c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:34.554172 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15634924-3e3a-4f09-b30c-7d469832116c service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:34.554577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15634924-3e3a-4f09-b30c-7d469832116c service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:34.554953 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15634924-3e3a-4f09-b30c-7d469832116c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:34.556765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-15634924-3e3a-4f09-b30c-7d469832116c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:34.583812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:34.675962 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15634924-3e3a-4f09-b30c-7d469832116c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:34.677768 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15634924-3e3a-4f09-b30c-7d469832116c service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:34.697170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 667/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:34:37.084826 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870fc03-cfc1-4811-ba66-c79d9a84bcc1 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:37.085435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870fc03-cfc1-4811-ba66-c79d9a84bcc1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:37.085435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870fc03-cfc1-4811-ba66-c79d9a84bcc1 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:37.087154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 697/1364] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:37.092448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1b39e-634f-4b11-a587-f813647baf36 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:37.092952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1b39e-634f-4b11-a587-f813647baf36 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:37.093344 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1b39e-634f-4b11-a587-f813647baf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:37.169565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-58c1b39e-634f-4b11-a587-f813647baf36 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:37.170834 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-58c1b39e-634f-4b11-a587-f813647baf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:37.285418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:37.294010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 668/1365] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:37.333313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a0e32-cc2a-4e37-94fa-d50c28ff378f None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:37.333584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a0e32-cc2a-4e37-94fa-d50c28ff378f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:37.334261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a0e32-cc2a-4e37-94fa-d50c28ff378f None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:37.336016 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 698/1366] 10.222.0.52 () {54 vars in 961 bytes} [Tue Jul 12 13:34:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:37.342082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e59633-dc3d-4e94-83f0-7a6c6d56fe85 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:37.342374 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e59633-dc3d-4e94-83f0-7a6c6d56fe85 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:37.342669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e59633-dc3d-4e94-83f0-7a6c6d56fe85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:37.422438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-65e59633-dc3d-4e94-83f0-7a6c6d56fe85 None None] MFA Rules not processed for user `0d3915cde0d04bcfb9b5ce8c89af1e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:37.424263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-65e59633-dc3d-4e94-83f0-7a6c6d56fe85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:37.580704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 669/1367] 10.222.0.52 () {58 vars in 1058 bytes} [Tue Jul 12 13:34:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:37.592172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a0e32-cc2a-4e37-94fa-d50c28ff378f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:37.593860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1d9a0e32-cc2a-4e37-94fa-d50c28ff378f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:37.766241 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a0e32-cc2a-4e37-94fa-d50c28ff378f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:37.790619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:37.927492 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00609d3-e304-4d0a-a7ca-820994c91c40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:37.928602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00609d3-e304-4d0a-a7ca-820994c91c40 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:37.929031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00609d3-e304-4d0a-a7ca-820994c91c40 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:37.929302 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00609d3-e304-4d0a-a7ca-820994c91c40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:37.931427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d00609d3-e304-4d0a-a7ca-820994c91c40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:38.047488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00609d3-e304-4d0a-a7ca-820994c91c40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:38.049114 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00609d3-e304-4d0a-a7ca-820994c91c40 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:38.069495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 699/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:34:38.976724 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b702cd4-498e-419e-bb67-b4f9e1776c5c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:38.977266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b702cd4-498e-419e-bb67-b4f9e1776c5c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:38.977266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b702cd4-498e-419e-bb67-b4f9e1776c5c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:38.982024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 670/1369] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:38.984358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c43187-d8b4-43c0-8a78-c184924de667 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:38.984885 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c43187-d8b4-43c0-8a78-c184924de667 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:38.985107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c43187-d8b4-43c0-8a78-c184924de667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:39.022741 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:39.064635 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-40c43187-d8b4-43c0-8a78-c184924de667 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:39.066003 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-40c43187-d8b4-43c0-8a78-c184924de667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:39.178470 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 700/1370] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:39.531912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35582b09-7b99-4896-9e55-6193c9b938bd None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:39.531912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35582b09-7b99-4896-9e55-6193c9b938bd None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:39.531912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35582b09-7b99-4896-9e55-6193c9b938bd None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:39.536902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 671/1371] 10.222.0.52 () {54 vars in 963 bytes} [Tue Jul 12 13:34:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:39.539322 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88bf82-a287-45c5-bc22-70d69b2a6f0e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:39.539511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88bf82-a287-45c5-bc22-70d69b2a6f0e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:39.540270 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88bf82-a287-45c5-bc22-70d69b2a6f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:39.623137 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-fa88bf82-a287-45c5-bc22-70d69b2a6f0e None None] MFA Rules not processed for user `35336384bbca475082be349df406ae3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:39.624635 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fa88bf82-a287-45c5-bc22-70d69b2a6f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:39.692793 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:39.728418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 701/1372] 10.222.0.52 () {58 vars in 1060 bytes} [Tue Jul 12 13:34:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:39.736724 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35582b09-7b99-4896-9e55-6193c9b938bd None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:39.738563 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-35582b09-7b99-4896-9e55-6193c9b938bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:39.798600 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:39.864741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35582b09-7b99-4896-9e55-6193c9b938bd None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:39.948686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20aa34c-79b4-455a-b8c9-6b62e50e1565 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:39.949403 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20aa34c-79b4-455a-b8c9-6b62e50e1565 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:39.949810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20aa34c-79b4-455a-b8c9-6b62e50e1565 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:39.950011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20aa34c-79b4-455a-b8c9-6b62e50e1565 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:39.951759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c20aa34c-79b4-455a-b8c9-6b62e50e1565 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:40.057934 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:40.075595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20aa34c-79b4-455a-b8c9-6b62e50e1565 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:40.077053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20aa34c-79b4-455a-b8c9-6b62e50e1565 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:40.097262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 672/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:34:40.201495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457882da-c311-4418-9b85-617a37d1af51 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:40.201495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457882da-c311-4418-9b85-617a37d1af51 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:40.201495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457882da-c311-4418-9b85-617a37d1af51 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:40.201495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 702/1374] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:40.207571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac931fb5-1798-44d3-a1bd-89ad35bc52db None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:40.207866 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac931fb5-1798-44d3-a1bd-89ad35bc52db None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:40.208500 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac931fb5-1798-44d3-a1bd-89ad35bc52db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:40.297362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ac931fb5-1798-44d3-a1bd-89ad35bc52db None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:40.299187 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ac931fb5-1798-44d3-a1bd-89ad35bc52db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:40.406680 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 673/1375] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:40.450602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457882da-c311-4418-9b85-617a37d1af51 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:40.452132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-457882da-c311-4418-9b85-617a37d1af51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:40.607356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457882da-c311-4418-9b85-617a37d1af51 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:40.659542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:40.723386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ac96e-f6b6-43c9-bffd-cb6fd2b91307 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:40.724044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ac96e-f6b6-43c9-bffd-cb6fd2b91307 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:40.724329 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ac96e-f6b6-43c9-bffd-cb6fd2b91307 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:40.724608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ac96e-f6b6-43c9-bffd-cb6fd2b91307 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:40.727000 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ab7ac96e-f6b6-43c9-bffd-cb6fd2b91307 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:40.822092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ac96e-f6b6-43c9-bffd-cb6fd2b91307 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:40.823682 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ac96e-f6b6-43c9-bffd-cb6fd2b91307 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:40.843745 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 703/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:34:42.920594 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299eac2-a0c5-4c3a-899b-7d52e800d8d1 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:42.921133 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299eac2-a0c5-4c3a-899b-7d52e800d8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:42.921133 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299eac2-a0c5-4c3a-899b-7d52e800d8d1 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:42.923124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 674/1377] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:42.928554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5126c02-a33b-46c2-aa35-3b1ea361f883 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:42.928988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5126c02-a33b-46c2-aa35-3b1ea361f883 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:42.929156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5126c02-a33b-46c2-aa35-3b1ea361f883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:42.995251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a5126c02-a33b-46c2-aa35-3b1ea361f883 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:42.996577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a5126c02-a33b-46c2-aa35-3b1ea361f883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:43.003939 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:43.101445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 704/1378] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:43.156465 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299eac2-a0c5-4c3a-899b-7d52e800d8d1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:43.158599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c299eac2-a0c5-4c3a-899b-7d52e800d8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:43.228233 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:43.320100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299eac2-a0c5-4c3a-899b-7d52e800d8d1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:43.433598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3600ae4-5f0c-4b0d-8a6f-fdf847197d65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:43.434287 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3600ae4-5f0c-4b0d-8a6f-fdf847197d65 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:43.434567 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3600ae4-5f0c-4b0d-8a6f-fdf847197d65 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:43.434838 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3600ae4-5f0c-4b0d-8a6f-fdf847197d65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:43.436470 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c3600ae4-5f0c-4b0d-8a6f-fdf847197d65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:43.455304 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:43.551791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3600ae4-5f0c-4b0d-8a6f-fdf847197d65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:43.553399 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3600ae4-5f0c-4b0d-8a6f-fdf847197d65 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:43.571992 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 675/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:34:45.711529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c9d16-ca31-4896-ba09-6828ba3485a3 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:45.712048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c9d16-ca31-4896-ba09-6828ba3485a3 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:45.712048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c9d16-ca31-4896-ba09-6828ba3485a3 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:45.713509 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 705/1380] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:45.718577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd70661-2bdf-48ed-a6cb-67108d9d3544 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:45.719023 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd70661-2bdf-48ed-a6cb-67108d9d3544 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:45.719135 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd70661-2bdf-48ed-a6cb-67108d9d3544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:45.799295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-fbd70661-2bdf-48ed-a6cb-67108d9d3544 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:45.800511 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fbd70661-2bdf-48ed-a6cb-67108d9d3544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:45.926674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:45.935194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 676/1381] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:45.978793 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c9d16-ca31-4896-ba09-6828ba3485a3 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:45.981097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4a9c9d16-ca31-4896-ba09-6828ba3485a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:46.156921 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:46.169599 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c9d16-ca31-4896-ba09-6828ba3485a3 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:46.298767 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c564c57-a214-4900-bd5c-72140bb37e7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:46.299337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c564c57-a214-4900-bd5c-72140bb37e7c service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:46.299625 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c564c57-a214-4900-bd5c-72140bb37e7c service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:46.300007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c564c57-a214-4900-bd5c-72140bb37e7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:46.301556 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8c564c57-a214-4900-bd5c-72140bb37e7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:46.424634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c564c57-a214-4900-bd5c-72140bb37e7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:46.427159 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c564c57-a214-4900-bd5c-72140bb37e7c service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:46.444961 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:46.461511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 706/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:34:48.582974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3cf063-d477-4859-80f0-18af7c792704 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:48.583526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3cf063-d477-4859-80f0-18af7c792704 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:48.583624 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3cf063-d477-4859-80f0-18af7c792704 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:48.585075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 677/1383] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:48.591467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaad4e2-ba0a-44dc-bc53-4ad80d005859 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:48.591908 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaad4e2-ba0a-44dc-bc53-4ad80d005859 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:48.592015 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaad4e2-ba0a-44dc-bc53-4ad80d005859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:48.660127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-abaad4e2-ba0a-44dc-bc53-4ad80d005859 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:48.661349 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-abaad4e2-ba0a-44dc-bc53-4ad80d005859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:48.766844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 707/1384] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:48.825277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3cf063-d477-4859-80f0-18af7c792704 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:48.827017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5d3cf063-d477-4859-80f0-18af7c792704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:48.978370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3cf063-d477-4859-80f0-18af7c792704 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:49.053152 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:49.103053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc721a-b1be-4e3f-9e77-a7403d0337b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:49.103627 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc721a-b1be-4e3f-9e77-a7403d0337b3 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:49.103917 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc721a-b1be-4e3f-9e77-a7403d0337b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:49.104186 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc721a-b1be-4e3f-9e77-a7403d0337b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:49.105831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0acc721a-b1be-4e3f-9e77-a7403d0337b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:49.205423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc721a-b1be-4e3f-9e77-a7403d0337b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:49.207071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc721a-b1be-4e3f-9e77-a7403d0337b3 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:49.224877 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 678/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:34:51.400853 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3ca1e-26ff-4cf1-bdcb-3b982527112a None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:51.402887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3ca1e-26ff-4cf1-bdcb-3b982527112a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:51.402887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3ca1e-26ff-4cf1-bdcb-3b982527112a None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:51.403279 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 708/1386] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:34:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:51.409517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df971e-4df8-42ca-b04d-9e1f3d2a1061 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:51.410240 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df971e-4df8-42ca-b04d-9e1f3d2a1061 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:51.410240 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df971e-4df8-42ca-b04d-9e1f3d2a1061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:51.489835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-69df971e-4df8-42ca-b04d-9e1f3d2a1061 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:51.491552 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-69df971e-4df8-42ca-b04d-9e1f3d2a1061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:51.544624 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:51.641803 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 679/1387] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:51.697674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3ca1e-26ff-4cf1-bdcb-3b982527112a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:51.699293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b6f3ca1e-26ff-4cf1-bdcb-3b982527112a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:51.780759 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:51.870092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3ca1e-26ff-4cf1-bdcb-3b982527112a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:51.991339 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aad1b1-78df-4208-8763-e8f7974e70e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:51.991910 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aad1b1-78df-4208-8763-e8f7974e70e0 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:51.992199 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aad1b1-78df-4208-8763-e8f7974e70e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:51.992479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aad1b1-78df-4208-8763-e8f7974e70e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:51.994164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a0aad1b1-78df-4208-8763-e8f7974e70e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:52.034634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:52.088801 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aad1b1-78df-4208-8763-e8f7974e70e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:52.090462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aad1b1-78df-4208-8763-e8f7974e70e0 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:52.109748 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 709/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:34:54.299176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c8f5b-f8db-47b5-a603-88da6bf2e0ea None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:54.299776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c8f5b-f8db-47b5-a603-88da6bf2e0ea None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:54.299776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c8f5b-f8db-47b5-a603-88da6bf2e0ea None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:54.301131 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 680/1389] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:34:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:54.307829 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a079a22-baa1-4ed2-ac51-c73841c72ba0 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:54.308684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a079a22-baa1-4ed2-ac51-c73841c72ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:54.308854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a079a22-baa1-4ed2-ac51-c73841c72ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:54.382646 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-6a079a22-baa1-4ed2-ac51-c73841c72ba0 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:54.384045 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6a079a22-baa1-4ed2-ac51-c73841c72ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:54.498448 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 710/1390] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:54.546288 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c8f5b-f8db-47b5-a603-88da6bf2e0ea None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:54.547820 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-376c8f5b-f8db-47b5-a603-88da6bf2e0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:54.706367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c8f5b-f8db-47b5-a603-88da6bf2e0ea None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:54.723372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:54.821808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcef2a-49c2-42e9-b019-2b62ff485cbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:54.822494 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcef2a-49c2-42e9-b019-2b62ff485cbe service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:54.822774 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcef2a-49c2-42e9-b019-2b62ff485cbe service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:54.823048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcef2a-49c2-42e9-b019-2b62ff485cbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:54.824717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-74fcef2a-49c2-42e9-b019-2b62ff485cbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:54.918390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fcef2a-49c2-42e9-b019-2b62ff485cbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:54.920034 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fcef2a-49c2-42e9-b019-2b62ff485cbe service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:54.938796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 681/1391] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:34:55.505427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cd6d0-24f6-4b66-90e7-e2f2fa24f1a9 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:55.506024 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cd6d0-24f6-4b66-90e7-e2f2fa24f1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:55.506024 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cd6d0-24f6-4b66-90e7-e2f2fa24f1a9 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:55.507677 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 711/1392] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:55.512556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb8b6d-0cca-4648-8d3d-a76100a25e14 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:55.512966 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb8b6d-0cca-4648-8d3d-a76100a25e14 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:55.513113 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb8b6d-0cca-4648-8d3d-a76100a25e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:55.539177 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:55.584430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-31cb8b6d-0cca-4648-8d3d-a76100a25e14 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:55.585769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-31cb8b6d-0cca-4648-8d3d-a76100a25e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:55.684218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 682/1393] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:55.740020 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9443d-74da-4cd8-a287-13171efa259b None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:55.740309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9443d-74da-4cd8-a287-13171efa259b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:55.740583 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9443d-74da-4cd8-a287-13171efa259b None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:55.741929 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 712/1394] 10.222.0.52 () {54 vars in 962 bytes} [Tue Jul 12 13:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:55.746982 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033ca28-c1a7-4ada-b1b4-24134723429e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:55.747271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033ca28-c1a7-4ada-b1b4-24134723429e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:55.747543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033ca28-c1a7-4ada-b1b4-24134723429e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:55.821690 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-c033ca28-c1a7-4ada-b1b4-24134723429e None None] MFA Rules not processed for user `ebb4da7b00c64fafbb3724bae25b2853`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:55.822841 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c033ca28-c1a7-4ada-b1b4-24134723429e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:55.867820 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:55.903944 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 683/1395] 10.222.0.52 () {58 vars in 1059 bytes} [Tue Jul 12 13:34:55 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:34:55.912053 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9443d-74da-4cd8-a287-13171efa259b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:55.913597 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0ac9443d-74da-4cd8-a287-13171efa259b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:55.927213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:56.022461 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9443d-74da-4cd8-a287-13171efa259b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:56.102272 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cd990-3a95-4b96-92f1-7e8845970f98 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebb4da7b00c64fafbb3724bae25b2853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:56.102927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cd990-3a95-4b96-92f1-7e8845970f98 service glance] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:56.103209 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cd990-3a95-4b96-92f1-7e8845970f98 service glance] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:56.103485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cd990-3a95-4b96-92f1-7e8845970f98 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:56.105135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-723cd990-3a95-4b96-92f1-7e8845970f98 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:56.179989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:56.223676 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cd990-3a95-4b96-92f1-7e8845970f98 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:56.225140 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cd990-3a95-4b96-92f1-7e8845970f98 service glance] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:56.244376 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 713/1396] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:34:56.962670 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90972e0-d1c9-4fc2-bb6a-8f01c8945b3d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:56.963132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90972e0-d1c9-4fc2-bb6a-8f01c8945b3d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:56.963262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90972e0-d1c9-4fc2-bb6a-8f01c8945b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:57.024854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c2d1a-8cfa-40ff-a309-f978f1132036 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:57.025419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c2d1a-8cfa-40ff-a309-f978f1132036 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:57.025419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c2d1a-8cfa-40ff-a309-f978f1132036 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:57.026811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 714/1397] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:57.032358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a47034-cf82-4213-b8a7-1d455b2102f1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:57.032523 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f90972e0-d1c9-4fc2-bb6a-8f01c8945b3d None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:57.032631 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a47034-cf82-4213-b8a7-1d455b2102f1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:57.032915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a47034-cf82-4213-b8a7-1d455b2102f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:57.033935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f90972e0-d1c9-4fc2-bb6a-8f01c8945b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:57.097601 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-45a47034-cf82-4213-b8a7-1d455b2102f1 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:57.098887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-45a47034-cf82-4213-b8a7-1d455b2102f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:57.122491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 684/1398] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jul 12 13:34:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:57.165670 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363b997-77d1-4dd1-9433-01886d2162d0 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:57.165977 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363b997-77d1-4dd1-9433-01886d2162d0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:57.166289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363b997-77d1-4dd1-9433-01886d2162d0 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:57.167685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 685/1399] 10.222.0.52 () {54 vars in 942 bytes} [Tue Jul 12 13:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:34:57.174473 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d8c35-c788-43f5-b8f7-57c96a97c74d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:57.174758 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d8c35-c788-43f5-b8f7-57c96a97c74d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:57.175034 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d8c35-c788-43f5-b8f7-57c96a97c74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:57.201279 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 715/1400] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:34:57.246398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a47034-cf82-4213-b8a7-1d455b2102f1 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:57.246577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:57.247755 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-45a47034-cf82-4213-b8a7-1d455b2102f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:57.255019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d56d8c35-c788-43f5-b8f7-57c96a97c74d None None] MFA Rules not processed for user `937bacab8c1c45c0876b8e61f6f3b859`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:57.256258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d56d8c35-c788-43f5-b8f7-57c96a97c74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:57.273980 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:57.364607 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 686/1401] 10.222.0.52 () {58 vars in 1039 bytes} [Tue Jul 12 13:34:57 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:34:57.366604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a47034-cf82-4213-b8a7-1d455b2102f1 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:57.374377 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d8c35-c788-43f5-b8f7-57c96a97c74d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:57.383957 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d8c35-c788-43f5-b8f7-57c96a97c74d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:57.494792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e82737-3d74-43bb-a4f4-63961a760772 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:57.495468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e82737-3d74-43bb-a4f4-63961a760772 service swift] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:57.495618 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e82737-3d74-43bb-a4f4-63961a760772 service swift] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:57.495889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e82737-3d74-43bb-a4f4-63961a760772 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:57.497576 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b1e82737-3d74-43bb-a4f4-63961a760772 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:57.497702 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01678b6-fc56-4412-8359-bdedfb865df5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:57.498374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01678b6-fc56-4412-8359-bdedfb865df5 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:57.498674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01678b6-fc56-4412-8359-bdedfb865df5 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:57.499204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01678b6-fc56-4412-8359-bdedfb865df5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:57.500488 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:57.500854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d01678b6-fc56-4412-8359-bdedfb865df5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:57.520008 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:57.602252 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e82737-3d74-43bb-a4f4-63961a760772 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:57.603911 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e82737-3d74-43bb-a4f4-63961a760772 service swift] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:57.613272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 687/1402] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:34:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 13:34:57.626237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01678b6-fc56-4412-8359-bdedfb865df5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:57.627880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01678b6-fc56-4412-8359-bdedfb865df5 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:57.647833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 716/1403] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:34:59.807886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb45db-07e7-475e-9330-40b92fe22484 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:59.808499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb45db-07e7-475e-9330-40b92fe22484 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:59.808499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb45db-07e7-475e-9330-40b92fe22484 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:59.814256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 688/1404] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:34:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:34:59.816551 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517218b4-8391-498b-8caf-f3d081c3fe81 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:59.817023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517218b4-8391-498b-8caf-f3d081c3fe81 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:59.817140 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517218b4-8391-498b-8caf-f3d081c3fe81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:59.907154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-517218b4-8391-498b-8caf-f3d081c3fe81 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:59.908416 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-517218b4-8391-498b-8caf-f3d081c3fe81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:00.019476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:00.025882 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 717/1405] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:00.075382 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb45db-07e7-475e-9330-40b92fe22484 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:00.076976 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a8fb45db-07e7-475e-9330-40b92fe22484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:00.242360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:00.246144 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb45db-07e7-475e-9330-40b92fe22484 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:00.378082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27c70a-55b0-47a8-b8bc-71da753f7917 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:00.378866 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27c70a-55b0-47a8-b8bc-71da753f7917 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:00.379222 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27c70a-55b0-47a8-b8bc-71da753f7917 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:00.379570 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27c70a-55b0-47a8-b8bc-71da753f7917 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:00.381484 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6c27c70a-55b0-47a8-b8bc-71da753f7917 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:00.484199 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27c70a-55b0-47a8-b8bc-71da753f7917 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:00.486049 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27c70a-55b0-47a8-b8bc-71da753f7917 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:00.508411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 689/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:02.701835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e42a0-ae4c-474f-855a-37a360516c4e None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:02.702460 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e42a0-ae4c-474f-855a-37a360516c4e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:02.702551 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e42a0-ae4c-474f-855a-37a360516c4e None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:02.704350 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 718/1407] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:02.709488 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decd5f2-7137-4da3-9894-730932e2cd6c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:02.710017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decd5f2-7137-4da3-9894-730932e2cd6c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:02.710122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decd5f2-7137-4da3-9894-730932e2cd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:02.713251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:02.780512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-5decd5f2-7137-4da3-9894-730932e2cd6c None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:02.782234 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5decd5f2-7137-4da3-9894-730932e2cd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:02.909998 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 690/1408] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:02.952941 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e42a0-ae4c-474f-855a-37a360516c4e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:02.954605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5d9e42a0-ae4c-474f-855a-37a360516c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:03.137727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e42a0-ae4c-474f-855a-37a360516c4e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:03.215111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:03.269245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284b32b-b3f9-4e5c-ae16-d6e3a6fbfba3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:03.269905 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284b32b-b3f9-4e5c-ae16-d6e3a6fbfba3 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:03.270220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284b32b-b3f9-4e5c-ae16-d6e3a6fbfba3 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:03.270500 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284b32b-b3f9-4e5c-ae16-d6e3a6fbfba3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:03.272172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8284b32b-b3f9-4e5c-ae16-d6e3a6fbfba3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:03.373362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284b32b-b3f9-4e5c-ae16-d6e3a6fbfba3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:03.375337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284b32b-b3f9-4e5c-ae16-d6e3a6fbfba3 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:03.399224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 719/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:12.904195 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=64 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:13.392628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=64 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:20.275669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c3cbf-bb0f-4659-b378-cc62115cc129 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:20.276223 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c3cbf-bb0f-4659-b378-cc62115cc129 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:20.276223 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c3cbf-bb0f-4659-b378-cc62115cc129 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:20.277595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 691/1410] 10.222.0.52 () {54 vars in 918 bytes} [Tue Jul 12 13:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:20.283363 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2f7de-e247-4305-88ee-40942d6d0f4c None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:20.283774 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2f7de-e247-4305-88ee-40942d6d0f4c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:20.283925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2f7de-e247-4305-88ee-40942d6d0f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:20.358006 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9ac2f7de-e247-4305-88ee-40942d6d0f4c None None] MFA Rules not processed for user `f1308deea1ea4629a1ac22d4b903ad4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:20.360173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9ac2f7de-e247-4305-88ee-40942d6d0f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:20.466291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 720/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Tue Jul 12 13:35:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:20.481214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259cf81-f60e-4abb-8e29-57dbcf8f0416 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:20.481497 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259cf81-f60e-4abb-8e29-57dbcf8f0416 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:20.481810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259cf81-f60e-4abb-8e29-57dbcf8f0416 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:20.483218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 692/1412] 10.222.0.52 () {54 vars in 918 bytes} [Tue Jul 12 13:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:20.488462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6e14b-6491-46a8-8f87-279cb935fa22 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:20.488755 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6e14b-6491-46a8-8f87-279cb935fa22 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:20.489038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6e14b-6491-46a8-8f87-279cb935fa22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:20.560349 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:20.563100 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-2cd6e14b-6491-46a8-8f87-279cb935fa22 None None] MFA Rules not processed for user `0d3915cde0d04bcfb9b5ce8c89af1e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:20.564956 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2cd6e14b-6491-46a8-8f87-279cb935fa22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:20.679711 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 721/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Tue Jul 12 13:35:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:23.184961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06af01-c1d1-4862-ade0-ee4cd8020c47 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:23.185531 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06af01-c1d1-4862-ade0-ee4cd8020c47 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:23.185531 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06af01-c1d1-4862-ade0-ee4cd8020c47 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:23.186905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 693/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:23.192716 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e94df-a8ff-457e-b813-a133ba174e92 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:23.193146 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e94df-a8ff-457e-b813-a133ba174e92 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:23.193276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e94df-a8ff-457e-b813-a133ba174e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:23.257815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-240e94df-a8ff-457e-b813-a133ba174e92 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:23.259093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-240e94df-a8ff-457e-b813-a133ba174e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:23.313347 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:23.355407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 722/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:23.397595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06af01-c1d1-4862-ade0-ee4cd8020c47 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:23.399171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ce06af01-c1d1-4862-ade0-ee4cd8020c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:23.546397 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06af01-c1d1-4862-ade0-ee4cd8020c47 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:23.619076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:23.659965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47613904-1418-44f6-abc8-72e7e9254641 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:23.660610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47613904-1418-44f6-abc8-72e7e9254641 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:23.660885 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47613904-1418-44f6-abc8-72e7e9254641 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:23.661157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47613904-1418-44f6-abc8-72e7e9254641 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:23.662844 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-47613904-1418-44f6-abc8-72e7e9254641 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:23.750540 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47613904-1418-44f6-abc8-72e7e9254641 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:23.752144 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47613904-1418-44f6-abc8-72e7e9254641 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:23.770183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 694/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:24.666089 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702950c-ee7a-48ee-9d70-fa6cee9467e0 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:24.666699 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702950c-ee7a-48ee-9d70-fa6cee9467e0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:24.666892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702950c-ee7a-48ee-9d70-fa6cee9467e0 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:24.668438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 723/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:35:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:24.674549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82aa11-3e1d-4c3f-a6fc-09e4f5bca2ce None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:24.675069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82aa11-3e1d-4c3f-a6fc-09e4f5bca2ce None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:24.675354 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82aa11-3e1d-4c3f-a6fc-09e4f5bca2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:24.749979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1b82aa11-3e1d-4c3f-a6fc-09e4f5bca2ce None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:24.751209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1b82aa11-3e1d-4c3f-a6fc-09e4f5bca2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:24.778446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:24.847273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 695/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:24.998849 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702950c-ee7a-48ee-9d70-fa6cee9467e0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:25.000540 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c702950c-ee7a-48ee-9d70-fa6cee9467e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:25.146972 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702950c-ee7a-48ee-9d70-fa6cee9467e0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:25.186202 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:25.260237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60c691-4faa-40ee-8251-7b82b8e1f2cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:25.260800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60c691-4faa-40ee-8251-7b82b8e1f2cc service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:25.261087 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60c691-4faa-40ee-8251-7b82b8e1f2cc service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:25.261366 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60c691-4faa-40ee-8251-7b82b8e1f2cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:25.263078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4a60c691-4faa-40ee-8251-7b82b8e1f2cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:25.353918 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60c691-4faa-40ee-8251-7b82b8e1f2cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:25.355575 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60c691-4faa-40ee-8251-7b82b8e1f2cc service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:25.373877 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 724/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:26.251404 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050f8c5-cb49-4921-9da7-4d212e7f708d None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:26.251982 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050f8c5-cb49-4921-9da7-4d212e7f708d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:26.251982 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050f8c5-cb49-4921-9da7-4d212e7f708d None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:26.253318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 696/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:26.258347 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500b298-e3df-400e-861c-271fdf6b8076 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:26.258778 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500b298-e3df-400e-861c-271fdf6b8076 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:26.258938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500b298-e3df-400e-861c-271fdf6b8076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:26.311429 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:26.333348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c500b298-e3df-400e-861c-271fdf6b8076 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:26.334601 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c500b298-e3df-400e-861c-271fdf6b8076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:26.430030 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 725/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:26.583122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050f8c5-cb49-4921-9da7-4d212e7f708d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:26.584501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8050f8c5-cb49-4921-9da7-4d212e7f708d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:26.731221 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050f8c5-cb49-4921-9da7-4d212e7f708d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:26.743147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:26.846025 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd05c10-4b6c-4d02-af9d-99a91cc89900 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:26.847114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd05c10-4b6c-4d02-af9d-99a91cc89900 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:26.847114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd05c10-4b6c-4d02-af9d-99a91cc89900 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:26.847300 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd05c10-4b6c-4d02-af9d-99a91cc89900 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:26.848812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-abd05c10-4b6c-4d02-af9d-99a91cc89900 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:26.938050 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd05c10-4b6c-4d02-af9d-99a91cc89900 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:26.939680 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd05c10-4b6c-4d02-af9d-99a91cc89900 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:26.958234 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 697/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:27.821064 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6239ac-9a30-4946-a471-593b6851468c None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:27.821593 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6239ac-9a30-4946-a471-593b6851468c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:27.821593 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6239ac-9a30-4946-a471-593b6851468c None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:27.822946 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 726/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:27.827971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52a3f2-3585-4e0e-a437-0779c397aac6 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:27.828359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52a3f2-3585-4e0e-a437-0779c397aac6 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:27.828532 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52a3f2-3585-4e0e-a437-0779c397aac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:27.850266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:27.907038 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7b52a3f2-3585-4e0e-a437-0779c397aac6 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:27.908258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7b52a3f2-3585-4e0e-a437-0779c397aac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:28.002961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 698/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:28.153814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6239ac-9a30-4946-a471-593b6851468c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:28.155351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9a6239ac-9a30-4946-a471-593b6851468c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:28.262942 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:28.300482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6239ac-9a30-4946-a471-593b6851468c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:28.411711 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4014ec-0a71-4cfb-957e-c7476725629e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:28.412278 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4014ec-0a71-4cfb-957e-c7476725629e service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:28.412562 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4014ec-0a71-4cfb-957e-c7476725629e service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:28.412837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4014ec-0a71-4cfb-957e-c7476725629e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:28.414515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2d4014ec-0a71-4cfb-957e-c7476725629e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:28.489653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:28.503902 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4014ec-0a71-4cfb-957e-c7476725629e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:28.505573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4014ec-0a71-4cfb-957e-c7476725629e service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:28.523254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 727/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:29.387329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df75d3a-e0fc-49e1-be75-e3d9cf6b1f5e None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:29.387873 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df75d3a-e0fc-49e1-be75-e3d9cf6b1f5e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:29.387873 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df75d3a-e0fc-49e1-be75-e3d9cf6b1f5e None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:29.389285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 699/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:29.395092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6eca09-0e65-43fb-8b31-242dbf8936c0 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:29.395593 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6eca09-0e65-43fb-8b31-242dbf8936c0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:29.395899 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6eca09-0e65-43fb-8b31-242dbf8936c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:29.470029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-cc6eca09-0e65-43fb-8b31-242dbf8936c0 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:29.471254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cc6eca09-0e65-43fb-8b31-242dbf8936c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:29.567441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 728/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:29.719099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df75d3a-e0fc-49e1-be75-e3d9cf6b1f5e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:29.720597 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9df75d3a-e0fc-49e1-be75-e3d9cf6b1f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:29.805971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:29.869159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df75d3a-e0fc-49e1-be75-e3d9cf6b1f5e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:29.986884 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922c013-fb4c-4588-ba68-ddf83ff6aaf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:29.987461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922c013-fb4c-4588-ba68-ddf83ff6aaf2 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:29.987748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922c013-fb4c-4588-ba68-ddf83ff6aaf2 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:29.988027 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922c013-fb4c-4588-ba68-ddf83ff6aaf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:29.989688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d922c013-fb4c-4588-ba68-ddf83ff6aaf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:30.037637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:30.079329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922c013-fb4c-4588-ba68-ddf83ff6aaf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:30.080981 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922c013-fb4c-4588-ba68-ddf83ff6aaf2 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:30.099373 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 700/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:30.905589 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe825d-54a2-44ee-aaf5-caacfdd755af None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:30.906279 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe825d-54a2-44ee-aaf5-caacfdd755af None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:30.906279 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe825d-54a2-44ee-aaf5-caacfdd755af None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:30.907587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 729/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:35:30.912527 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776e8d0-5e71-460d-a9ee-b3269287d143 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:30.912960 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776e8d0-5e71-460d-a9ee-b3269287d143 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:30.913079 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776e8d0-5e71-460d-a9ee-b3269287d143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:30.986339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-5776e8d0-5e71-460d-a9ee-b3269287d143 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:30.987558 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5776e8d0-5e71-460d-a9ee-b3269287d143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:31.085231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 701/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:31.241042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe825d-54a2-44ee-aaf5-caacfdd755af None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:31.242638 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-69fe825d-54a2-44ee-aaf5-caacfdd755af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:31.291392 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:31.394025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe825d-54a2-44ee-aaf5-caacfdd755af None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:31.506584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadabca-5084-4206-9e7b-f586e9d66adb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:31.507143 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadabca-5084-4206-9e7b-f586e9d66adb service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:31.507462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadabca-5084-4206-9e7b-f586e9d66adb service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:31.507703 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadabca-5084-4206-9e7b-f586e9d66adb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:31.509658 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1fadabca-5084-4206-9e7b-f586e9d66adb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:31.516953 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:31.602239 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadabca-5084-4206-9e7b-f586e9d66adb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:31.603863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadabca-5084-4206-9e7b-f586e9d66adb service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:31.622568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 730/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:32.425326 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe56c3-ef76-4e4a-b3ce-0ee5d9bb4062 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:32.425909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe56c3-ef76-4e4a-b3ce-0ee5d9bb4062 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:32.425909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe56c3-ef76-4e4a-b3ce-0ee5d9bb4062 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:32.427311 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 702/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:32.432354 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4006f7-0465-4cd5-a8fc-0ffdefffe1e0 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:32.432802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4006f7-0465-4cd5-a8fc-0ffdefffe1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:32.432919 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4006f7-0465-4cd5-a8fc-0ffdefffe1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:32.511587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8d4006f7-0465-4cd5-a8fc-0ffdefffe1e0 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:32.512817 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8d4006f7-0465-4cd5-a8fc-0ffdefffe1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:32.582120 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:32.612455 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 731/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:32.766181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe56c3-ef76-4e4a-b3ce-0ee5d9bb4062 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:32.767600 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1fbe56c3-ef76-4e4a-b3ce-0ee5d9bb4062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:32.786274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:32.920042 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe56c3-ef76-4e4a-b3ce-0ee5d9bb4062 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:33.006072 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:33.032415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afca696-bc46-4acc-9207-593a7940f920 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:33.033020 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afca696-bc46-4acc-9207-593a7940f920 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:33.033306 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afca696-bc46-4acc-9207-593a7940f920 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:33.033584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afca696-bc46-4acc-9207-593a7940f920 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:33.035286 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6afca696-bc46-4acc-9207-593a7940f920 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:33.122140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afca696-bc46-4acc-9207-593a7940f920 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:33.123741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afca696-bc46-4acc-9207-593a7940f920 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:33.145218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 703/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:33.949156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e142b0-5892-4d43-8434-5815ed3a2166 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:33.949719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e142b0-5892-4d43-8434-5815ed3a2166 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:33.949828 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e142b0-5892-4d43-8434-5815ed3a2166 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:33.951217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 732/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:33.956194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0596c-4769-434f-98a9-ae120c6e33b8 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:33.956625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0596c-4769-434f-98a9-ae120c6e33b8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:33.956733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0596c-4769-434f-98a9-ae120c6e33b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:34.029570 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4dd0596c-4769-434f-98a9-ae120c6e33b8 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:34.031340 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4dd0596c-4769-434f-98a9-ae120c6e33b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:34.073076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:34.128988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 704/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:34.278087 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e142b0-5892-4d43-8434-5815ed3a2166 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:34.279642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-82e142b0-5892-4d43-8434-5815ed3a2166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:34.424651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e142b0-5892-4d43-8434-5815ed3a2166 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:34.478261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:34.537646 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d53786-1e16-4bae-bb7a-475d610bf9b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:34.538326 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d53786-1e16-4bae-bb7a-475d610bf9b9 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:34.538614 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d53786-1e16-4bae-bb7a-475d610bf9b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:34.538894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d53786-1e16-4bae-bb7a-475d610bf9b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:34.540529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-33d53786-1e16-4bae-bb7a-475d610bf9b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:34.631094 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d53786-1e16-4bae-bb7a-475d610bf9b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:34.632706 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d53786-1e16-4bae-bb7a-475d610bf9b9 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:34.651059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 733/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:35.459519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc7303-40a5-4c88-9cbf-6b92022706d1 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:35.460090 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc7303-40a5-4c88-9cbf-6b92022706d1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:35.460090 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc7303-40a5-4c88-9cbf-6b92022706d1 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:35.461476 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 705/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:35.466346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f05de-f666-4566-adec-7f8750373856 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:35.466773 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f05de-f666-4566-adec-7f8750373856 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:35.466906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f05de-f666-4566-adec-7f8750373856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:35.543798 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a15f05de-f666-4566-adec-7f8750373856 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:35.545020 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a15f05de-f666-4566-adec-7f8750373856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:35.547083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:35.644222 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 734/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:35.794796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc7303-40a5-4c88-9cbf-6b92022706d1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:35.796228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-12cc7303-40a5-4c88-9cbf-6b92022706d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:35.944101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc7303-40a5-4c88-9cbf-6b92022706d1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:35.970768 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:36.056956 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9f73f-c740-4052-9995-9fef26a449ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:36.057531 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9f73f-c740-4052-9995-9fef26a449ed service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:36.057811 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9f73f-c740-4052-9995-9fef26a449ed service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:36.058116 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9f73f-c740-4052-9995-9fef26a449ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:36.059983 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c3e9f73f-c740-4052-9995-9fef26a449ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:36.149436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9f73f-c740-4052-9995-9fef26a449ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:36.151084 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9f73f-c740-4052-9995-9fef26a449ed service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:36.169285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 706/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:38.112642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a823a-ce81-445c-a519-1156cba9df9d None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:38.113186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a823a-ce81-445c-a519-1156cba9df9d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:38.113186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a823a-ce81-445c-a519-1156cba9df9d None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:38.114614 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 735/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:38.119962 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f108aa-e72a-44ba-a939-25230b5e68e2 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:38.120359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f108aa-e72a-44ba-a939-25230b5e68e2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:38.120527 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f108aa-e72a-44ba-a939-25230b5e68e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:38.160165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:38.185014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a8f108aa-e72a-44ba-a939-25230b5e68e2 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:38.186264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a8f108aa-e72a-44ba-a939-25230b5e68e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:38.282205 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 707/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:38.320182 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a823a-ce81-445c-a519-1156cba9df9d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:38.321739 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a68a823a-ce81-445c-a519-1156cba9df9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:38.427424 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a823a-ce81-445c-a519-1156cba9df9d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:38.459444 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:38.499212 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5bee5-c25a-4fd0-ac25-88c73f689958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:38.500656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5bee5-c25a-4fd0-ac25-88c73f689958 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:38.500936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5bee5-c25a-4fd0-ac25-88c73f689958 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:38.501209 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5bee5-c25a-4fd0-ac25-88c73f689958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:38.502934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-06d5bee5-c25a-4fd0-ac25-88c73f689958 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:38.611364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5bee5-c25a-4fd0-ac25-88c73f689958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:38.612815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5bee5-c25a-4fd0-ac25-88c73f689958 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:38.632021 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 736/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:40.932394 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6be4b0-d8ef-4ea8-8c18-4e1a83b11f02 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:40.932999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6be4b0-d8ef-4ea8-8c18-4e1a83b11f02 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:40.932999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6be4b0-d8ef-4ea8-8c18-4e1a83b11f02 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:40.934443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 708/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:40.939757 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5082a-b1f6-4e9b-bc0f-17126d221b4e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:40.940187 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5082a-b1f6-4e9b-bc0f-17126d221b4e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:40.940325 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5082a-b1f6-4e9b-bc0f-17126d221b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:41.004652 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-e2b5082a-b1f6-4e9b-bc0f-17126d221b4e None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:41.005873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e2b5082a-b1f6-4e9b-bc0f-17126d221b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:41.022009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:41.108103 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 737/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:41.146336 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6be4b0-d8ef-4ea8-8c18-4e1a83b11f02 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:41.147941 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1e6be4b0-d8ef-4ea8-8c18-4e1a83b11f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:41.249808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6be4b0-d8ef-4ea8-8c18-4e1a83b11f02 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:41.261591 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:41.324030 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6feb95e-c367-4c2b-9e78-f6efed19c99a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:41.324664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6feb95e-c367-4c2b-9e78-f6efed19c99a service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:41.324943 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6feb95e-c367-4c2b-9e78-f6efed19c99a service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:41.325213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6feb95e-c367-4c2b-9e78-f6efed19c99a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:41.326882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e6feb95e-c367-4c2b-9e78-f6efed19c99a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:41.433566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6feb95e-c367-4c2b-9e78-f6efed19c99a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:41.434962 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6feb95e-c367-4c2b-9e78-f6efed19c99a service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:41.453710 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 709/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:43.784673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4372bc9-1cbc-4605-891f-c5821d9717ca None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:43.785283 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4372bc9-1cbc-4605-891f-c5821d9717ca None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:43.785283 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4372bc9-1cbc-4605-891f-c5821d9717ca None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:43.786619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 738/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:43.792060 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e1fe4-c58b-4696-85f5-c58193654221 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:43.792512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e1fe4-c58b-4696-85f5-c58193654221 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:43.792629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e1fe4-c58b-4696-85f5-c58193654221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:43.839170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:43.858788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0c1e1fe4-c58b-4696-85f5-c58193654221 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:43.860014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0c1e1fe4-c58b-4696-85f5-c58193654221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:43.956001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 710/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:43.994209 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4372bc9-1cbc-4605-891f-c5821d9717ca None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:43.995852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c4372bc9-1cbc-4605-891f-c5821d9717ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:44.101354 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4372bc9-1cbc-4605-891f-c5821d9717ca None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:44.143215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:44.176832 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14344326-ba1b-40eb-b3ae-b358d7ac6fcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:44.177484 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14344326-ba1b-40eb-b3ae-b358d7ac6fcd service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:44.177768 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14344326-ba1b-40eb-b3ae-b358d7ac6fcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:44.178094 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14344326-ba1b-40eb-b3ae-b358d7ac6fcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:44.179756 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-14344326-ba1b-40eb-b3ae-b358d7ac6fcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:44.408667 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14344326-ba1b-40eb-b3ae-b358d7ac6fcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:44.410059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14344326-ba1b-40eb-b3ae-b358d7ac6fcd service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:44.427726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 739/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:46.585971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76234174-64ed-438b-9b9b-270c95673cd7 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:46.586801 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76234174-64ed-438b-9b9b-270c95673cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:46.586801 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76234174-64ed-438b-9b9b-270c95673cd7 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:46.587879 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 711/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:46.593744 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94870c1-e8b8-4e51-80e3-e30cd39d526a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:46.594256 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94870c1-e8b8-4e51-80e3-e30cd39d526a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:46.594352 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94870c1-e8b8-4e51-80e3-e30cd39d526a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:46.657793 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a94870c1-e8b8-4e51-80e3-e30cd39d526a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:46.659040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a94870c1-e8b8-4e51-80e3-e30cd39d526a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:46.680606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:46.761492 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 740/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:35:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:46.800168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76234174-64ed-438b-9b9b-270c95673cd7 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:46.801712 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-76234174-64ed-438b-9b9b-270c95673cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:46.908589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76234174-64ed-438b-9b9b-270c95673cd7 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:46.928319 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:46.982101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d144d6-ae96-44c0-b9eb-430912f76f1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:46.982749 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d144d6-ae96-44c0-b9eb-430912f76f1b service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:46.983035 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d144d6-ae96-44c0-b9eb-430912f76f1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:46.983315 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d144d6-ae96-44c0-b9eb-430912f76f1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:46.985002 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-63d144d6-ae96-44c0-b9eb-430912f76f1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:47.098076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d144d6-ae96-44c0-b9eb-430912f76f1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:47.099447 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d144d6-ae96-44c0-b9eb-430912f76f1b service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:47.117430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 712/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:49.289315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf249ed-547b-458a-80e1-5dff01dc05e9 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:49.289886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf249ed-547b-458a-80e1-5dff01dc05e9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:49.289886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf249ed-547b-458a-80e1-5dff01dc05e9 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:49.291259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 741/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:49.297051 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed6058-1d0e-49af-9774-af2ee65a750c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:49.297454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed6058-1d0e-49af-9774-af2ee65a750c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:49.297616 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed6058-1d0e-49af-9774-af2ee65a750c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:49.359360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:49.362165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b4ed6058-1d0e-49af-9774-af2ee65a750c None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:49.363985 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b4ed6058-1d0e-49af-9774-af2ee65a750c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:49.457917 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 713/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:49.496147 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf249ed-547b-458a-80e1-5dff01dc05e9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:49.497669 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-faf249ed-547b-458a-80e1-5dff01dc05e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:49.600283 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf249ed-547b-458a-80e1-5dff01dc05e9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:49.648860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:49.674349 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53505a-b285-470b-bef1-0b4f99614460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:49.674990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53505a-b285-470b-bef1-0b4f99614460 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:49.675267 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53505a-b285-470b-bef1-0b4f99614460 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:49.675539 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53505a-b285-470b-bef1-0b4f99614460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:49.677181 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ec53505a-b285-470b-bef1-0b4f99614460 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:49.784255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53505a-b285-470b-bef1-0b4f99614460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:49.785785 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53505a-b285-470b-bef1-0b4f99614460 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:49.804751 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 742/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:51.897482 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce098d6-762d-421e-8d67-4fe33a0f6481 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:51.898057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce098d6-762d-421e-8d67-4fe33a0f6481 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:51.898057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce098d6-762d-421e-8d67-4fe33a0f6481 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:51.899487 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 714/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:51.904794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ef18d-1b0d-4465-8e94-71f38b263574 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:51.905183 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ef18d-1b0d-4465-8e94-71f38b263574 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:51.905862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ef18d-1b0d-4465-8e94-71f38b263574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:51.969908 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b89ef18d-1b0d-4465-8e94-71f38b263574 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:51.971153 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b89ef18d-1b0d-4465-8e94-71f38b263574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:52.000860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:52.068357 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 743/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:52.106938 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce098d6-762d-421e-8d67-4fe33a0f6481 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:52.108477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1ce098d6-762d-421e-8d67-4fe33a0f6481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:52.209886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce098d6-762d-421e-8d67-4fe33a0f6481 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:52.238613 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:52.286286 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557be09-7069-4e97-a64c-cfe5d9d36e55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:52.286940 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557be09-7069-4e97-a64c-cfe5d9d36e55 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:52.287224 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557be09-7069-4e97-a64c-cfe5d9d36e55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:52.287507 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557be09-7069-4e97-a64c-cfe5d9d36e55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:52.289187 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7557be09-7069-4e97-a64c-cfe5d9d36e55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:52.397587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557be09-7069-4e97-a64c-cfe5d9d36e55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:52.398970 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557be09-7069-4e97-a64c-cfe5d9d36e55 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:52.417463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 715/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:54.496546 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a6528-c2f1-4c18-90f1-fb969059187a None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:54.497111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a6528-c2f1-4c18-90f1-fb969059187a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:54.497229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a6528-c2f1-4c18-90f1-fb969059187a None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:54.498559 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 744/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:54.504181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602acc0c-8887-491b-b79d-0056dc4493ce None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:54.504572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602acc0c-8887-491b-b79d-0056dc4493ce None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:54.504734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602acc0c-8887-491b-b79d-0056dc4493ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:54.570115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-602acc0c-8887-491b-b79d-0056dc4493ce None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:54.571359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-602acc0c-8887-491b-b79d-0056dc4493ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:54.580614 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:54.674399 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 716/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:54.727268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a6528-c2f1-4c18-90f1-fb969059187a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:54.728839 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3e4a6528-c2f1-4c18-90f1-fb969059187a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:54.835233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a6528-c2f1-4c18-90f1-fb969059187a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:54.890781 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:54.908875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45209df8-73a5-4983-a751-f69a30440f13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:54.909523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45209df8-73a5-4983-a751-f69a30440f13 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:54.909805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45209df8-73a5-4983-a751-f69a30440f13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:54.910107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45209df8-73a5-4983-a751-f69a30440f13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:54.911740 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-45209df8-73a5-4983-a751-f69a30440f13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:55.019241 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45209df8-73a5-4983-a751-f69a30440f13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:55.020683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45209df8-73a5-4983-a751-f69a30440f13 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:55.038417 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 745/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:57.122117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2471130-f416-43e4-babf-bac9620f40ad None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:57.122676 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2471130-f416-43e4-babf-bac9620f40ad None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:57.122676 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2471130-f416-43e4-babf-bac9620f40ad None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:57.124076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 717/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:57.129452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabd46c-f706-4f81-bb3a-ef864780ad38 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:57.129902 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabd46c-f706-4f81-bb3a-ef864780ad38 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:57.130041 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabd46c-f706-4f81-bb3a-ef864780ad38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:57.193269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9fabd46c-f706-4f81-bb3a-ef864780ad38 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:57.194503 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9fabd46c-f706-4f81-bb3a-ef864780ad38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:57.231798 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:57.292831 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 746/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:57.330857 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2471130-f416-43e4-babf-bac9620f40ad None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:57.332398 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e2471130-f416-43e4-babf-bac9620f40ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:57.433855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2471130-f416-43e4-babf-bac9620f40ad None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:57.471004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:57.506028 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe746d-cca8-4470-9fd7-6bfcfad0d6bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:57.506678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe746d-cca8-4470-9fd7-6bfcfad0d6bc service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:57.506960 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe746d-cca8-4470-9fd7-6bfcfad0d6bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:57.507237 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe746d-cca8-4470-9fd7-6bfcfad0d6bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:57.508884 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2cfe746d-cca8-4470-9fd7-6bfcfad0d6bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:57.624808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe746d-cca8-4470-9fd7-6bfcfad0d6bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:57.626280 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe746d-cca8-4470-9fd7-6bfcfad0d6bc service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:57.644500 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 718/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:59.717859 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d358279-8b83-476d-bc98-137d2f180906 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:59.718491 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d358279-8b83-476d-bc98-137d2f180906 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:59.718491 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d358279-8b83-476d-bc98-137d2f180906 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:59.719892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 747/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:35:59.725494 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca99995-95bb-4dd2-9b04-91fc9590c834 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:59.725997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca99995-95bb-4dd2-9b04-91fc9590c834 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:59.726115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca99995-95bb-4dd2-9b04-91fc9590c834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:59.800374 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9ca99995-95bb-4dd2-9b04-91fc9590c834 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:59.801632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9ca99995-95bb-4dd2-9b04-91fc9590c834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:59.822507 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:59.923992 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 719/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:59.961583 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d358279-8b83-476d-bc98-137d2f180906 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:59.963257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5d358279-8b83-476d-bc98-137d2f180906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:00.077267 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d358279-8b83-476d-bc98-137d2f180906 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:00.152500 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:00.160341 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d759f-0abe-482a-9ace-8e2f97e53c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:00.160995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d759f-0abe-482a-9ace-8e2f97e53c3d service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:00.161277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d759f-0abe-482a-9ace-8e2f97e53c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:00.161550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d759f-0abe-482a-9ace-8e2f97e53c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:00.163241 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4c0d759f-0abe-482a-9ace-8e2f97e53c3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:00.270235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d759f-0abe-482a-9ace-8e2f97e53c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:00.271703 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d759f-0abe-482a-9ace-8e2f97e53c3d service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:00.291706 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 748/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:36:02.356709 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3e6a5-7889-4fcf-873c-e748432f7cb0 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:02.357382 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3e6a5-7889-4fcf-873c-e748432f7cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:02.357382 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3e6a5-7889-4fcf-873c-e748432f7cb0 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:02.358761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 720/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:36:02.364098 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1768beb-b957-49d8-b9df-67ff2b277e21 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:02.364527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1768beb-b957-49d8-b9df-67ff2b277e21 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:02.364641 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1768beb-b957-49d8-b9df-67ff2b277e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:02.430920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-e1768beb-b957-49d8-b9df-67ff2b277e21 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:02.432297 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e1768beb-b957-49d8-b9df-67ff2b277e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:02.483467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:02.535485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 749/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:02.571143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3e6a5-7889-4fcf-873c-e748432f7cb0 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:02.572719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-50b3e6a5-7889-4fcf-873c-e748432f7cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:02.680139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3e6a5-7889-4fcf-873c-e748432f7cb0 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:02.727912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:02.755996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929f0ed-91b9-4d8c-9d7d-8ab9e34ebd7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:02.756642 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929f0ed-91b9-4d8c-9d7d-8ab9e34ebd7f service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:02.756934 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929f0ed-91b9-4d8c-9d7d-8ab9e34ebd7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:02.757209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929f0ed-91b9-4d8c-9d7d-8ab9e34ebd7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:02.758920 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1929f0ed-91b9-4d8c-9d7d-8ab9e34ebd7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:02.865175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929f0ed-91b9-4d8c-9d7d-8ab9e34ebd7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:02.866696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929f0ed-91b9-4d8c-9d7d-8ab9e34ebd7f service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:02.885164 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 721/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:36:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:36:12.533003 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=22 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:12.882874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=66 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:22.540407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cd2ec-0c69-4b40-b0c0-2678bf9de7ec None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:22.541020 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cd2ec-0c69-4b40-b0c0-2678bf9de7ec None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:22.541020 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cd2ec-0c69-4b40-b0c0-2678bf9de7ec None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:22.542382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 750/1471] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:36:22.547484 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0f234-7c6c-4f39-86f7-7023906f8bec None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:22.547920 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0f234-7c6c-4f39-86f7-7023906f8bec None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:22.548051 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0f234-7c6c-4f39-86f7-7023906f8bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:22.611111 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3db0f234-7c6c-4f39-86f7-7023906f8bec None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:22.612298 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3db0f234-7c6c-4f39-86f7-7023906f8bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:22.710130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 722/1472] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:22.794074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7363ae5-0844-4c50-83fa-d55c70e59496 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:22.794309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7363ae5-0844-4c50-83fa-d55c70e59496 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:22.794615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7363ae5-0844-4c50-83fa-d55c70e59496 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:22.795979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 751/1473] 10.222.0.52 () {54 vars in 962 bytes} [Tue Jul 12 13:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:36:22.801137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddcb0d-b661-47d0-819c-ebd321a7b7f0 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:22.801416 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddcb0d-b661-47d0-819c-ebd321a7b7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:22.801700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddcb0d-b661-47d0-819c-ebd321a7b7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:22.883732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-06ddcb0d-b661-47d0-819c-ebd321a7b7f0 None None] MFA Rules not processed for user `ebb4da7b00c64fafbb3724bae25b2853`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:22.884974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-06ddcb0d-b661-47d0-819c-ebd321a7b7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:22.906184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:22.962843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 723/1474] 10.222.0.52 () {58 vars in 1059 bytes} [Tue Jul 12 13:36:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:22.970463 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7363ae5-0844-4c50-83fa-d55c70e59496 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:22.971982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b7363ae5-0844-4c50-83fa-d55c70e59496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:23.076978 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7363ae5-0844-4c50-83fa-d55c70e59496 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:23.149867 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805a568-5a95-414c-8c5c-b77c1b4b156d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebb4da7b00c64fafbb3724bae25b2853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:23.150541 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805a568-5a95-414c-8c5c-b77c1b4b156d service glance] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:23.150830 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805a568-5a95-414c-8c5c-b77c1b4b156d service glance] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:23.151110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805a568-5a95-414c-8c5c-b77c1b4b156d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:23.152760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f805a568-5a95-414c-8c5c-b77c1b4b156d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:23.205343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:23.259618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805a568-5a95-414c-8c5c-b77c1b4b156d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:23.261073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805a568-5a95-414c-8c5c-b77c1b4b156d service glance] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:23.278901 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 752/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:36:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:36:25.377455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87a04a-8585-46e0-a533-824339f6f5e8 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:25.378082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87a04a-8585-46e0-a533-824339f6f5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:25.378082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87a04a-8585-46e0-a533-824339f6f5e8 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:25.379438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 724/1476] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:36:25.384550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82e03e-df41-4871-9577-040218977a96 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:25.385019 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82e03e-df41-4871-9577-040218977a96 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:25.385127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82e03e-df41-4871-9577-040218977a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:25.449146 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-bc82e03e-df41-4871-9577-040218977a96 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:25.450409 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-bc82e03e-df41-4871-9577-040218977a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:25.547540 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 753/1477] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:25.570775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87a04a-8585-46e0-a533-824339f6f5e8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:25.572290 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5b87a04a-8585-46e0-a533-824339f6f5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:25.695120 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87a04a-8585-46e0-a533-824339f6f5e8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:25.742155 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:25.786951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7633e9-2a8b-484e-981f-0cfbc401d896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:25.787609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7633e9-2a8b-484e-981f-0cfbc401d896 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:25.787892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7633e9-2a8b-484e-981f-0cfbc401d896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:25.788166 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7633e9-2a8b-484e-981f-0cfbc401d896 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:25.788883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7633e9-2a8b-484e-981f-0cfbc401d896 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:25.790704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: WARNING py.warnings [None req-3c7633e9-2a8b-484e-981f-0cfbc401d896 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:36:25.790704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: warnings.warn(msg) Jul 12 13:36:25.790704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: Jul 12 13:36:25.791838 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7633e9-2a8b-484e-981f-0cfbc401d896 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:25.810195 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 725/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jul 12 13:36:25 2022] GET /identity/v3/projects => generated 2553 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:36:27.850248 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f35507-afb6-4157-8775-0e3a75e68a24 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:27.850757 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f35507-afb6-4157-8775-0e3a75e68a24 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:27.850884 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f35507-afb6-4157-8775-0e3a75e68a24 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:27.852174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 754/1479] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:36:27.857496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c1b3e-522b-4adc-a9b4-03d73f2842f0 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:27.857927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c1b3e-522b-4adc-a9b4-03d73f2842f0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:27.858160 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c1b3e-522b-4adc-a9b4-03d73f2842f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:27.922147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0e0c1b3e-522b-4adc-a9b4-03d73f2842f0 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:27.923352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0e0c1b3e-522b-4adc-a9b4-03d73f2842f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:28.138375 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 726/1480] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:28.180550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f35507-afb6-4157-8775-0e3a75e68a24 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:28.182150 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-94f35507-afb6-4157-8775-0e3a75e68a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:28.192535 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:28.325403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f35507-afb6-4157-8775-0e3a75e68a24 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:28.403102 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:28.436422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b4355-a914-4153-8af5-a59703273d1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:28.437067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b4355-a914-4153-8af5-a59703273d1b service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:28.437355 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b4355-a914-4153-8af5-a59703273d1b service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:28.437635 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b4355-a914-4153-8af5-a59703273d1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:28.439327 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-981b4355-a914-4153-8af5-a59703273d1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:28.526677 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b4355-a914-4153-8af5-a59703273d1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:28.528266 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b4355-a914-4153-8af5-a59703273d1b service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:28.548251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 755/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:36:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:36:30.746899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db75ce-8215-49d0-a5d4-e1036bc7d831 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:30.747470 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db75ce-8215-49d0-a5d4-e1036bc7d831 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:30.747470 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db75ce-8215-49d0-a5d4-e1036bc7d831 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:30.749418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 727/1482] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:36:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:36:30.754929 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ecb6d-46bd-42d9-a7f3-ae6cc1f10b59 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:30.755373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ecb6d-46bd-42d9-a7f3-ae6cc1f10b59 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:30.755470 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ecb6d-46bd-42d9-a7f3-ae6cc1f10b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:30.819784 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-371ecb6d-46bd-42d9-a7f3-ae6cc1f10b59 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:30.820991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-371ecb6d-46bd-42d9-a7f3-ae6cc1f10b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:30.868369 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:30.918816 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 756/1483] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:30.955166 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db75ce-8215-49d0-a5d4-e1036bc7d831 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:30.956742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-37db75ce-8215-49d0-a5d4-e1036bc7d831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:30.979524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:31.059776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db75ce-8215-49d0-a5d4-e1036bc7d831 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:31.133896 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158fa6e-e278-4b72-9d59-851b5b00e9f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:31.134573 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158fa6e-e278-4b72-9d59-851b5b00e9f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:31.134858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158fa6e-e278-4b72-9d59-851b5b00e9f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:31.135135 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158fa6e-e278-4b72-9d59-851b5b00e9f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:31.136785 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8158fa6e-e278-4b72-9d59-851b5b00e9f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:31.208479 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:31.243627 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158fa6e-e278-4b72-9d59-851b5b00e9f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:31.245062 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158fa6e-e278-4b72-9d59-851b5b00e9f3 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:31.263641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 728/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:36:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:36:33.350376 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4de8ed-cfe0-454c-82ac-8754235e29a5 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:33.350925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4de8ed-cfe0-454c-82ac-8754235e29a5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:33.350925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4de8ed-cfe0-454c-82ac-8754235e29a5 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:33.352296 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 757/1485] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:36:33.357441 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23356e62-dd25-444f-9549-2201f56e6193 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:33.357946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23356e62-dd25-444f-9549-2201f56e6193 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:33.358168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23356e62-dd25-444f-9549-2201f56e6193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:33.421074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-23356e62-dd25-444f-9549-2201f56e6193 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:33.422295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-23356e62-dd25-444f-9549-2201f56e6193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:33.519824 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 729/1486] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:33.557062 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4de8ed-cfe0-454c-82ac-8754235e29a5 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:33.558584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1c4de8ed-cfe0-454c-82ac-8754235e29a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:33.662158 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4de8ed-cfe0-454c-82ac-8754235e29a5 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:33.731850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:33.733621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd8ea0-71de-4cba-a39f-824b4663d0a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:33.734391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd8ea0-71de-4cba-a39f-824b4663d0a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:33.734929 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd8ea0-71de-4cba-a39f-824b4663d0a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:33.735412 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd8ea0-71de-4cba-a39f-824b4663d0a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:33.737293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-77fd8ea0-71de-4cba-a39f-824b4663d0a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:33.845191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd8ea0-71de-4cba-a39f-824b4663d0a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:33.846592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd8ea0-71de-4cba-a39f-824b4663d0a4 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:33.864891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 758/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:36:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:36:35.978652 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16602e33-490b-499f-912f-7e552d93088b None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:35.979177 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16602e33-490b-499f-912f-7e552d93088b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:35.979177 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16602e33-490b-499f-912f-7e552d93088b None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:35.980473 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 730/1488] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:36:35.986360 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce958d-d3f6-4fd3-9a28-3b6c8e23e630 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:35.986793 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce958d-d3f6-4fd3-9a28-3b6c8e23e630 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:35.986916 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce958d-d3f6-4fd3-9a28-3b6c8e23e630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:36.051291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-61ce958d-d3f6-4fd3-9a28-3b6c8e23e630 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:36.052514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-61ce958d-d3f6-4fd3-9a28-3b6c8e23e630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:36.106035 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:36.151370 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 759/1489] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:36.191723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16602e33-490b-499f-912f-7e552d93088b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:36.193239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-16602e33-490b-499f-912f-7e552d93088b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:36.228284 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:36.294549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16602e33-490b-499f-912f-7e552d93088b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:36.366108 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e48e9f4-68e3-4d35-bc00-3307546aa9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:36.366746 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48e9f4-68e3-4d35-bc00-3307546aa9da service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:36.367026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48e9f4-68e3-4d35-bc00-3307546aa9da service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:36.367298 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48e9f4-68e3-4d35-bc00-3307546aa9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:36.368901 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5e48e9f4-68e3-4d35-bc00-3307546aa9da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:36.445749 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:36.475649 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e48e9f4-68e3-4d35-bc00-3307546aa9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:36.477082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e48e9f4-68e3-4d35-bc00-3307546aa9da service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:36.494426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 731/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:36:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:36:38.666586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df76d89-e8cb-47d9-a6d4-57b4ee4615db None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:38.667172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df76d89-e8cb-47d9-a6d4-57b4ee4615db None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:38.667172 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df76d89-e8cb-47d9-a6d4-57b4ee4615db None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:38.668482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 760/1491] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jul 12 13:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:36:38.674506 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4675c-4858-489a-8b21-415968bd5ca6 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:38.674908 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4675c-4858-489a-8b21-415968bd5ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:38.675067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4675c-4858-489a-8b21-415968bd5ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:38.738336 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-94c4675c-4858-489a-8b21-415968bd5ca6 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:38.739545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-94c4675c-4858-489a-8b21-415968bd5ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:38.833584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 732/1492] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jul 12 13:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:38.871719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df76d89-e8cb-47d9-a6d4-57b4ee4615db None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:38.873945 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0df76d89-e8cb-47d9-a6d4-57b4ee4615db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:38.975408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df76d89-e8cb-47d9-a6d4-57b4ee4615db None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:39.046044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f758cf-ecf5-49c0-a73e-e23fc2d81e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:39.046680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f758cf-ecf5-49c0-a73e-e23fc2d81e58 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:39.046960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f758cf-ecf5-49c0-a73e-e23fc2d81e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:39.047233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f758cf-ecf5-49c0-a73e-e23fc2d81e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:39.048840 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-68f758cf-ecf5-49c0-a73e-e23fc2d81e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:39.057563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:39.153700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f758cf-ecf5-49c0-a73e-e23fc2d81e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:39.155687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f758cf-ecf5-49c0-a73e-e23fc2d81e58 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:39.173584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 761/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:36:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:36:48.831760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=82 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:49.171352 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=50 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:38:05.004799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fbbd1-55d5-4b0a-a690-5c814bfcfd61 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:05.005310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fbbd1-55d5-4b0a-a690-5c814bfcfd61 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:05.005418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fbbd1-55d5-4b0a-a690-5c814bfcfd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:05.079252 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-077fbbd1-55d5-4b0a-a690-5c814bfcfd61 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:38:05.080462 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-077fbbd1-55d5-4b0a-a690-5c814bfcfd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:05.175375 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 733/1494] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:38:05.183351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e93c5-6852-4959-9724-01153592fe8a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:05.183848 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e93c5-6852-4959-9724-01153592fe8a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:05.183961 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e93c5-6852-4959-9724-01153592fe8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:05.257751 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-616e93c5-6852-4959-9724-01153592fe8a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:38:05.258999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-616e93c5-6852-4959-9724-01153592fe8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:05.354225 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 762/1495] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:38:05.359125 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fbbd1-55d5-4b0a-a690-5c814bfcfd61 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:38:05.360323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-077fbbd1-55d5-4b0a-a690-5c814bfcfd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:05.372149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fbbd1-55d5-4b0a-a690-5c814bfcfd61 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:38:05.431584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:38:05.465584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1af69-b1d7-4520-b337-c73a23a51387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:38:05.466288 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1af69-b1d7-4520-b337-c73a23a51387 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:05.466571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1af69-b1d7-4520-b337-c73a23a51387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:05.466854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1af69-b1d7-4520-b337-c73a23a51387 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:05.467579 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1af69-b1d7-4520-b337-c73a23a51387 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:38:05.467884 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1af69-b1d7-4520-b337-c73a23a51387 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:38:05.468897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: WARNING py.warnings [None req-11c1af69-b1d7-4520-b337-c73a23a51387 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:38:05.468897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: warnings.warn(msg) Jul 12 13:38:05.468897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: Jul 12 13:38:05.469778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1af69-b1d7-4520-b337-c73a23a51387 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:38:05.481370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 734/1496] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:38:05.489601 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e93c5-6852-4959-9724-01153592fe8a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:38:05.499275 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e93c5-6852-4959-9724-01153592fe8a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:38:05.563802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:38:05.597613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264f424-7899-4741-bbc5-5ad7be8d00f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:38:05.598304 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264f424-7899-4741-bbc5-5ad7be8d00f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:05.598582 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264f424-7899-4741-bbc5-5ad7be8d00f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:05.598850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264f424-7899-4741-bbc5-5ad7be8d00f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:05.599520 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264f424-7899-4741-bbc5-5ad7be8d00f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:38:05.601243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: WARNING py.warnings [None req-4264f424-7899-4741-bbc5-5ad7be8d00f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:38:05.601243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: warnings.warn(msg) Jul 12 13:38:05.601243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: Jul 12 13:38:05.602444 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264f424-7899-4741-bbc5-5ad7be8d00f0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:38:05.656442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 763/1497] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:38:05 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:38:05.660780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1af69-b1d7-4520-b337-c73a23a51387 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:38:05.662452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-11c1af69-b1d7-4520-b337-c73a23a51387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:05.696117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1af69-b1d7-4520-b337-c73a23a51387 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:38:05.788048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd91b8e-506a-4f88-9910-935704a4fd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:38:05.788716 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd91b8e-506a-4f88-9910-935704a4fd38 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:05.788999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd91b8e-506a-4f88-9910-935704a4fd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:05.789277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd91b8e-506a-4f88-9910-935704a4fd38 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:05.789988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd91b8e-506a-4f88-9910-935704a4fd38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:38:05.791017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: WARNING py.warnings [None req-5dd91b8e-506a-4f88-9910-935704a4fd38 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:38:05.791017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: warnings.warn(msg) Jul 12 13:38:05.791017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: Jul 12 13:38:05.792233 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd91b8e-506a-4f88-9910-935704a4fd38 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:38:05.832753 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 735/1498] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:38:05 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:38:05.836890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264f424-7899-4741-bbc5-5ad7be8d00f0 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:38:05.838447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4264f424-7899-4741-bbc5-5ad7be8d00f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:05.901412 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264f424-7899-4741-bbc5-5ad7be8d00f0 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:38:05.982754 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:38:05.994625 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f9a3d-7d77-47a4-92db-bc42adff8c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:38:05.995289 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f9a3d-7d77-47a4-92db-bc42adff8c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:05.995569 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f9a3d-7d77-47a4-92db-bc42adff8c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:05.995865 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f9a3d-7d77-47a4-92db-bc42adff8c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:05.996620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f9a3d-7d77-47a4-92db-bc42adff8c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:38:05.997642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: WARNING py.warnings [None req-936f9a3d-7d77-47a4-92db-bc42adff8c59 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:38:05.997642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: warnings.warn(msg) Jul 12 13:38:05.997642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: Jul 12 13:38:05.998550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f9a3d-7d77-47a4-92db-bc42adff8c59 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:38:06.010957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 764/1499] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:38:06.015116 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd91b8e-506a-4f88-9910-935704a4fd38 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:38:06.016634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5dd91b8e-506a-4f88-9910-935704a4fd38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:06.028719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd91b8e-506a-4f88-9910-935704a4fd38 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:38:06.053844 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:38:06.126608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821eddb9-4ce8-450c-acc4-f4aee8728ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:38:06.127279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821eddb9-4ce8-450c-acc4-f4aee8728ace admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:06.127561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821eddb9-4ce8-450c-acc4-f4aee8728ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:06.127833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821eddb9-4ce8-450c-acc4-f4aee8728ace admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:06.128557 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821eddb9-4ce8-450c-acc4-f4aee8728ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:38:06.129570 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: WARNING py.warnings [None req-821eddb9-4ce8-450c-acc4-f4aee8728ace admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:38:06.129570 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: warnings.warn(msg) Jul 12 13:38:06.129570 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: Jul 12 13:38:06.130483 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821eddb9-4ce8-450c-acc4-f4aee8728ace admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:38:06.142513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 736/1500] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:38:06.146729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f9a3d-7d77-47a4-92db-bc42adff8c59 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:38:06.148269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-936f9a3d-7d77-47a4-92db-bc42adff8c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:06.181929 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f9a3d-7d77-47a4-92db-bc42adff8c59 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:38:06.273089 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4aeb7e-1d82-4e88-81a8-658457755e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:38:06.273903 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aeb7e-1d82-4e88-81a8-658457755e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:06.274221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aeb7e-1d82-4e88-81a8-658457755e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:06.274499 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aeb7e-1d82-4e88-81a8-658457755e4e admin admin] PATH_INFO: `/v3/projects/50796aa2327b4b14b1b1fd163b4764ed/users/aab7cab92bc749b181110e6ad0fe7160/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:06.297125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4aeb7e-1d82-4e88-81a8-658457755e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=50796aa2327b4b14b1b1fd163b4764ed, user_id=aab7cab92bc749b181110e6ad0fe7160, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:38:06.298200 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: WARNING py.warnings [None req-7c4aeb7e-1d82-4e88-81a8-658457755e4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:38:06.298200 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: warnings.warn(msg) Jul 12 13:38:06.298200 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: Jul 12 13:38:06.300134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4aeb7e-1d82-4e88-81a8-658457755e4e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:38:06.325795 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 765/1501] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:38:06 2022] PUT /identity/v3/projects/50796aa2327b4b14b1b1fd163b4764ed/users/aab7cab92bc749b181110e6ad0fe7160/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:38:06.337763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd0e3c-e35d-49de-9ac6-b267e2333b13 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:06.338100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd0e3c-e35d-49de-9ac6-b267e2333b13 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:06.338390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd0e3c-e35d-49de-9ac6-b267e2333b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:06.393646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-44cd0e3c-e35d-49de-9ac6-b267e2333b13 None None] MFA Rules not processed for user `aab7cab92bc749b181110e6ad0fe7160`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:38:06.394884 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-44cd0e3c-e35d-49de-9ac6-b267e2333b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:06.471449 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 737/1502] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:38:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:38:06.484677 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18b6df-8ac2-4f31-a120-ca1b481d8bab None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:06.484968 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18b6df-8ac2-4f31-a120-ca1b481d8bab None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:06.485244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18b6df-8ac2-4f31-a120-ca1b481d8bab None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:06.486655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 766/1503] 10.222.0.52 () {54 vars in 963 bytes} [Tue Jul 12 13:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:38:06.491441 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00318e-ad75-4f2b-87ab-6b321b7647f1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:06.491731 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00318e-ad75-4f2b-87ab-6b321b7647f1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:06.492026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00318e-ad75-4f2b-87ab-6b321b7647f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:06.521421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:38:06.565454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0b00318e-ad75-4f2b-87ab-6b321b7647f1 None None] MFA Rules not processed for user `35336384bbca475082be349df406ae3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:38:06.566613 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0b00318e-ad75-4f2b-87ab-6b321b7647f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:06.640248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 738/1504] 10.222.0.52 () {58 vars in 1060 bytes} [Tue Jul 12 13:38:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:38:06.647831 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18b6df-8ac2-4f31-a120-ca1b481d8bab None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:38:06.649357 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-dc18b6df-8ac2-4f31-a120-ca1b481d8bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:06.698506 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:38:06.753153 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18b6df-8ac2-4f31-a120-ca1b481d8bab None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:38:06.823992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303af78-7367-4c2a-9a2d-83fa3c83e909 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:38:06.825189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303af78-7367-4c2a-9a2d-83fa3c83e909 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:06.825460 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303af78-7367-4c2a-9a2d-83fa3c83e909 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:06.825733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303af78-7367-4c2a-9a2d-83fa3c83e909 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:06.827448 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f303af78-7367-4c2a-9a2d-83fa3c83e909 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:06.930954 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303af78-7367-4c2a-9a2d-83fa3c83e909 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:38:06.932405 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303af78-7367-4c2a-9a2d-83fa3c83e909 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:38:06.936480 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:38:06.951779 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 767/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:38:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:38:06.992403 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc7f22-26eb-4152-a372-4a4a7175f333 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:06.992717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc7f22-26eb-4152-a372-4a4a7175f333 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:06.993019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc7f22-26eb-4152-a372-4a4a7175f333 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:06.994352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 739/1506] 10.222.0.52 () {52 vars in 829 bytes} [Tue Jul 12 13:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:38:06.998420 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303af78-7367-4c2a-9a2d-83fa3c83e909 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:38:06.999612 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f303af78-7367-4c2a-9a2d-83fa3c83e909 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:07.097996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303af78-7367-4c2a-9a2d-83fa3c83e909 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:38:07.194210 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a2cfc-a40e-40de-912b-e41da72802d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:38:07.195086 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a2cfc-a40e-40de-912b-e41da72802d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:07.195398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a2cfc-a40e-40de-912b-e41da72802d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:07.195677 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a2cfc-a40e-40de-912b-e41da72802d9 admin admin] PATH_INFO: `/v3/users/aab7cab92bc749b181110e6ad0fe7160` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:07.207726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a2cfc-a40e-40de-912b-e41da72802d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aab7cab92bc749b181110e6ad0fe7160)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:38:07.208772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: WARNING py.warnings [None req-b07a2cfc-a40e-40de-912b-e41da72802d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:38:07.208772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: warnings.warn(msg) Jul 12 13:38:07.208772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: Jul 12 13:38:07.210016 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a2cfc-a40e-40de-912b-e41da72802d9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:38:07.219536 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:38:07.274784 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b07a2cfc-a40e-40de-912b-e41da72802d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aab7cab92bc749b181110e6ad0fe7160'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:38:07.275483 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b07a2cfc-a40e-40de-912b-e41da72802d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aab7cab92bc749b181110e6ad0fe7160'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:38:07.275736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b07a2cfc-a40e-40de-912b-e41da72802d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aab7cab92bc749b181110e6ad0fe7160'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:38:07.294133 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b07a2cfc-a40e-40de-912b-e41da72802d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aab7cab92bc749b181110e6ad0fe7160'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:38:07.306510 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b07a2cfc-a40e-40de-912b-e41da72802d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aab7cab92bc749b181110e6ad0fe7160'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:38:07.321570 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 768/1507] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:38:06 2022] DELETE /identity/v3/users/aab7cab92bc749b181110e6ad0fe7160 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:38:07.326165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9ec75-2622-43c9-9861-ae85ea7e61bf None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:07.326631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9ec75-2622-43c9-9861-ae85ea7e61bf None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:07.326785 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9ec75-2622-43c9-9861-ae85ea7e61bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:07.381278 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-cac9ec75-2622-43c9-9861-ae85ea7e61bf None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:38:07.382508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cac9ec75-2622-43c9-9861-ae85ea7e61bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:07.456614 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:38:07.469189 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 740/1508] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:38:07.478762 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a2cfc-a40e-40de-912b-e41da72802d9 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:38:07.479971 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b07a2cfc-a40e-40de-912b-e41da72802d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:07.577551 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a2cfc-a40e-40de-912b-e41da72802d9 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:38:07.630883 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:38:07.651472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97147fc6-b92d-4be7-a390-cbd4fdce3ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:38:07.652628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97147fc6-b92d-4be7-a390-cbd4fdce3ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:07.652916 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97147fc6-b92d-4be7-a390-cbd4fdce3ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:07.653196 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97147fc6-b92d-4be7-a390-cbd4fdce3ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:07.654871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-97147fc6-b92d-4be7-a390-cbd4fdce3ae4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:07.762140 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97147fc6-b92d-4be7-a390-cbd4fdce3ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:38:07.763644 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97147fc6-b92d-4be7-a390-cbd4fdce3ae4 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:38:07.783693 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 769/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:38:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:38:08.118830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9ec75-2622-43c9-9861-ae85ea7e61bf None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:38:08.120070 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cac9ec75-2622-43c9-9861-ae85ea7e61bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:08.132289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9ec75-2622-43c9-9861-ae85ea7e61bf None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:38:08.228920 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a37dbb-6eae-42c4-99a0-242e0ad10c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:38:08.229705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a37dbb-6eae-42c4-99a0-242e0ad10c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:08.230079 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a37dbb-6eae-42c4-99a0-242e0ad10c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:08.230389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a37dbb-6eae-42c4-99a0-242e0ad10c5e admin admin] PATH_INFO: `/v3/projects/50796aa2327b4b14b1b1fd163b4764ed` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:08.240143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a37dbb-6eae-42c4-99a0-242e0ad10c5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=50796aa2327b4b14b1b1fd163b4764ed)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:38:08.241179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: WARNING py.warnings [None req-55a37dbb-6eae-42c4-99a0-242e0ad10c5e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:38:08.241179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: warnings.warn(msg) Jul 12 13:38:08.241179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: Jul 12 13:38:08.242411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a37dbb-6eae-42c4-99a0-242e0ad10c5e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:38:08.299198 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-55a37dbb-6eae-42c4-99a0-242e0ad10c5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50796aa2327b4b14b1b1fd163b4764ed', 'request_id': 'req-55a37dbb-6eae-42c4-99a0-242e0ad10c5e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:38:08.305190 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-55a37dbb-6eae-42c4-99a0-242e0ad10c5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50796aa2327b4b14b1b1fd163b4764ed', 'request_id': 'req-55a37dbb-6eae-42c4-99a0-242e0ad10c5e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:38:08.315520 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-55a37dbb-6eae-42c4-99a0-242e0ad10c5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50796aa2327b4b14b1b1fd163b4764ed', 'request_id': 'req-55a37dbb-6eae-42c4-99a0-242e0ad10c5e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:38:08.344883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-55a37dbb-6eae-42c4-99a0-242e0ad10c5e admin admin] The token cache is being invalidate because project 50796aa2327b4b14b1b1fd163b4764ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:38:08.345496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-55a37dbb-6eae-42c4-99a0-242e0ad10c5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:38:08.345665 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-55a37dbb-6eae-42c4-99a0-242e0ad10c5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:38:08.347056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 741/1510] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:38:08 2022] DELETE /identity/v3/projects/50796aa2327b4b14b1b1fd163b4764ed => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:38:17.781298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=62 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:38:18.343068 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:39:51.631779 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaad5d-4ef0-4c09-a312-9a6493adf783 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:39:51.634175 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaad5d-4ef0-4c09-a312-9a6493adf783 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:39:51.634175 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaad5d-4ef0-4c09-a312-9a6493adf783 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:39:51.634390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 770/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:39:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:39:51.639435 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de27fb5-bbdd-48b4-bf5c-558eab35ca62 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:39:51.639896 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de27fb5-bbdd-48b4-bf5c-558eab35ca62 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:39:51.640010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de27fb5-bbdd-48b4-bf5c-558eab35ca62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:39:51.721008 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1de27fb5-bbdd-48b4-bf5c-558eab35ca62 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:39:51.722265 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1de27fb5-bbdd-48b4-bf5c-558eab35ca62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:39:51.825198 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 742/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:39:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:39:51.978846 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aaad5d-4ef0-4c09-a312-9a6493adf783 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:39:51.980946 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f8aaad5d-4ef0-4c09-a312-9a6493adf783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:39:52.100856 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aaad5d-4ef0-4c09-a312-9a6493adf783 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:39:52.107007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a293a6-b3ed-43e3-9b88-7bd628f84936 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:39:52.107657 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a293a6-b3ed-43e3-9b88-7bd628f84936 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:39:52.107944 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a293a6-b3ed-43e3-9b88-7bd628f84936 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:39:52.108223 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a293a6-b3ed-43e3-9b88-7bd628f84936 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:39:52.120443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a293a6-b3ed-43e3-9b88-7bd628f84936 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:39:52.122116 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a293a6-b3ed-43e3-9b88-7bd628f84936 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:39:52.132093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 771/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:39:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:39:52.981754 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c693a7d-0687-495d-bf46-15e559f91532 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:39:52.982354 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c693a7d-0687-495d-bf46-15e559f91532 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:39:52.982471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c693a7d-0687-495d-bf46-15e559f91532 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:39:52.983808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 743/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:39:52.988580 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315c0d1-7b07-4c84-940c-ad09cf9b3c2b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:39:52.989056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315c0d1-7b07-4c84-940c-ad09cf9b3c2b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:39:52.989277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315c0d1-7b07-4c84-940c-ad09cf9b3c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:39:53.007687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4315c0d1-7b07-4c84-940c-ad09cf9b3c2b None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:39:53.008889 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4315c0d1-7b07-4c84-940c-ad09cf9b3c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:39:53.017388 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 772/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:39:53.170738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c693a7d-0687-495d-bf46-15e559f91532 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:39:53.186200 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c693a7d-0687-495d-bf46-15e559f91532 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:39:53.191816 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeafbc6-9635-4a21-8b1c-4dc590e69ba7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:39:53.192457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeafbc6-9635-4a21-8b1c-4dc590e69ba7 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:39:53.192741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeafbc6-9635-4a21-8b1c-4dc590e69ba7 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:39:53.193021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeafbc6-9635-4a21-8b1c-4dc590e69ba7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:39:53.202522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeafbc6-9635-4a21-8b1c-4dc590e69ba7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:39:53.204109 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeafbc6-9635-4a21-8b1c-4dc590e69ba7 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:39:53.214648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 744/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:39:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:39:53.554952 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0d739-c8b1-49da-9b2f-108f878f5c78 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:39:53.555162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0d739-c8b1-49da-9b2f-108f878f5c78 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:39:53.555436 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0d739-c8b1-49da-9b2f-108f878f5c78 None None] PATH_INFO: `/v3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:39:53.557351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 773/1517] 10.222.0.50 () {54 vars in 925 bytes} [Tue Jul 12 13:39:53 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:39:53.563374 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9639f8b-88d8-47a3-93dd-c970dcb4c958 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:39:53.563667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9639f8b-88d8-47a3-93dd-c970dcb4c958 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:39:53.563945 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9639f8b-88d8-47a3-93dd-c970dcb4c958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:39:53.622376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a9639f8b-88d8-47a3-93dd-c970dcb4c958 None None] MFA Rules not processed for user `f1308deea1ea4629a1ac22d4b903ad4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:39:53.623588 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a9639f8b-88d8-47a3-93dd-c970dcb4c958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:39:53.640169 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:39:53.694590 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 745/1518] 10.222.0.50 () {58 vars in 1015 bytes} [Tue Jul 12 13:39:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:39:57.424601 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3bab2-158c-4ad6-9361-5121e0648435 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:39:57.425138 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3bab2-158c-4ad6-9361-5121e0648435 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:39:57.425138 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3bab2-158c-4ad6-9361-5121e0648435 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:39:57.426482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 774/1519] 10.222.0.52 () {54 vars in 960 bytes} [Tue Jul 12 13:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:39:57.431525 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cfa46-087e-466d-970b-b457af90b965 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:39:57.431972 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cfa46-087e-466d-970b-b457af90b965 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:39:57.432080 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cfa46-087e-466d-970b-b457af90b965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:39:57.450483 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b91cfa46-087e-466d-970b-b457af90b965 None None] MFA Rules not processed for user `f1308deea1ea4629a1ac22d4b903ad4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:39:57.451684 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b91cfa46-087e-466d-970b-b457af90b965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:39:57.460229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 746/1520] 10.222.0.52 () {58 vars in 1057 bytes} [Tue Jul 12 13:39:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:39:57.468225 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3bab2-158c-4ad6-9361-5121e0648435 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:39:57.481275 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3bab2-158c-4ad6-9361-5121e0648435 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:39:57.486173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6615a99-0802-4313-84dd-8ab939404407 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1308deea1ea4629a1ac22d4b903ad4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:39:57.486813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6615a99-0802-4313-84dd-8ab939404407 service placement] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:39:57.487093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6615a99-0802-4313-84dd-8ab939404407 service placement] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:39:57.487372 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6615a99-0802-4313-84dd-8ab939404407 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:39:57.495584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6615a99-0802-4313-84dd-8ab939404407 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:39:57.497158 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6615a99-0802-4313-84dd-8ab939404407 service placement] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:39:57.507014 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 775/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:39:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:40:04.470250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac0b6e-3bdc-44c2-9544-6df249006c07 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:04.470816 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac0b6e-3bdc-44c2-9544-6df249006c07 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:04.470816 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac0b6e-3bdc-44c2-9544-6df249006c07 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:04.472187 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 747/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:40:04.477061 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e905f7-d574-431b-b097-144e999c2461 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:04.477455 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e905f7-d574-431b-b097-144e999c2461 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:04.477629 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e905f7-d574-431b-b097-144e999c2461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:04.495939 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-80e905f7-d574-431b-b097-144e999c2461 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:40:04.497141 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-80e905f7-d574-431b-b097-144e999c2461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:40:04.505213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 776/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:40:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:40:04.658854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac0b6e-3bdc-44c2-9544-6df249006c07 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:40:04.674384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac0b6e-3bdc-44c2-9544-6df249006c07 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:40:04.679871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963778c4-7457-4bb8-8e80-88e66a40bed2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:40:04.680501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963778c4-7457-4bb8-8e80-88e66a40bed2 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:04.680788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963778c4-7457-4bb8-8e80-88e66a40bed2 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:04.681082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963778c4-7457-4bb8-8e80-88e66a40bed2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:04.689098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963778c4-7457-4bb8-8e80-88e66a40bed2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:40:04.690716 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963778c4-7457-4bb8-8e80-88e66a40bed2 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:40:04.700527 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 748/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:40:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:40:05.516409 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e99431-caa9-47ae-b528-215aca9f7c4f None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:05.516950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e99431-caa9-47ae-b528-215aca9f7c4f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:05.516950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e99431-caa9-47ae-b528-215aca9f7c4f None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:05.518379 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 777/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:40:05.523210 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8cc24-3c7b-4a79-a2a1-68d27f178fb2 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:05.523652 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8cc24-3c7b-4a79-a2a1-68d27f178fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:05.523770 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8cc24-3c7b-4a79-a2a1-68d27f178fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:05.542133 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-97c8cc24-3c7b-4a79-a2a1-68d27f178fb2 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:40:05.543391 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-97c8cc24-3c7b-4a79-a2a1-68d27f178fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:40:05.551267 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 749/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:40:05.700580 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e99431-caa9-47ae-b528-215aca9f7c4f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:40:05.715015 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e99431-caa9-47ae-b528-215aca9f7c4f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:40:05.720424 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c830b1-2e6f-4d24-a268-c4cfa06771c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:40:05.721057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c830b1-2e6f-4d24-a268-c4cfa06771c6 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:05.721344 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c830b1-2e6f-4d24-a268-c4cfa06771c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:05.721622 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c830b1-2e6f-4d24-a268-c4cfa06771c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:05.730312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c830b1-2e6f-4d24-a268-c4cfa06771c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:40:05.731909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c830b1-2e6f-4d24-a268-c4cfa06771c6 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:40:05.741289 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 778/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:40:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:40:09.748405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d68c8-d446-4e60-8531-e4fde614ed59 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:09.748921 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d68c8-d446-4e60-8531-e4fde614ed59 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:09.749026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d68c8-d446-4e60-8531-e4fde614ed59 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:09.750409 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 750/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:40:09.755962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfd280-d58b-4898-8435-1a37474cd572 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:09.756400 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfd280-d58b-4898-8435-1a37474cd572 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:09.756504 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfd280-d58b-4898-8435-1a37474cd572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:09.774990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-81dfd280-d58b-4898-8435-1a37474cd572 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:40:09.776221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-81dfd280-d58b-4898-8435-1a37474cd572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:40:09.784026 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 779/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:40:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:40:09.857169 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d68c8-d446-4e60-8531-e4fde614ed59 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:40:09.858907 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a67d68c8-d446-4e60-8531-e4fde614ed59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:40:09.941932 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d68c8-d446-4e60-8531-e4fde614ed59 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:40:09.945586 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00968d5-141a-472a-83aa-f2daf075f484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:40:09.946266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00968d5-141a-472a-83aa-f2daf075f484 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:09.946553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00968d5-141a-472a-83aa-f2daf075f484 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:09.947483 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00968d5-141a-472a-83aa-f2daf075f484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:09.956455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00968d5-141a-472a-83aa-f2daf075f484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:40:09.957876 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00968d5-141a-472a-83aa-f2daf075f484 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:40:09.967359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 751/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:40:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:40:11.997704 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c5b9e-d5dc-49f9-bb83-07da1ef98ad0 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:11.998267 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c5b9e-d5dc-49f9-bb83-07da1ef98ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:11.998267 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c5b9e-d5dc-49f9-bb83-07da1ef98ad0 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:11.999576 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 780/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:40:12.005008 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10520a09-630f-4398-8256-a5dc54e85784 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:12.005443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10520a09-630f-4398-8256-a5dc54e85784 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:12.005589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10520a09-630f-4398-8256-a5dc54e85784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:12.023343 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-10520a09-630f-4398-8256-a5dc54e85784 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:40:12.024519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-10520a09-630f-4398-8256-a5dc54e85784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:40:12.032849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 752/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:40:12.068144 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c5b9e-d5dc-49f9-bb83-07da1ef98ad0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:40:12.078914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c5b9e-d5dc-49f9-bb83-07da1ef98ad0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:40:12.082372 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c86185-ae3f-4df0-8849-e05299cc2b54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:40:12.083004 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c86185-ae3f-4df0-8849-e05299cc2b54 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:12.083285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c86185-ae3f-4df0-8849-e05299cc2b54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:12.083557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c86185-ae3f-4df0-8849-e05299cc2b54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:12.092870 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c86185-ae3f-4df0-8849-e05299cc2b54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:40:12.094337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c86185-ae3f-4df0-8849-e05299cc2b54 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:40:12.104265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 781/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:40:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:40:15.755856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560d671-00fc-4607-b116-fe48e39a407e None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:15.756360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560d671-00fc-4607-b116-fe48e39a407e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:15.756471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560d671-00fc-4607-b116-fe48e39a407e None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:15.757847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 753/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:40:15.763079 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253177e-1cdd-47bd-a59a-50965bfa08f1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:15.763497 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253177e-1cdd-47bd-a59a-50965bfa08f1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:15.763613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253177e-1cdd-47bd-a59a-50965bfa08f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:15.781598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-f253177e-1cdd-47bd-a59a-50965bfa08f1 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:40:15.782809 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f253177e-1cdd-47bd-a59a-50965bfa08f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:40:15.791174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 782/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:40:16.094308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560d671-00fc-4607-b116-fe48e39a407e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:40:16.095969 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9560d671-00fc-4607-b116-fe48e39a407e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:40:16.159825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:40:16.173064 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560d671-00fc-4607-b116-fe48e39a407e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:40:16.176676 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cffcb30-c89d-4fec-864e-d8a46697c186 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:40:16.177317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cffcb30-c89d-4fec-864e-d8a46697c186 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:16.177601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cffcb30-c89d-4fec-864e-d8a46697c186 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:16.177879 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cffcb30-c89d-4fec-864e-d8a46697c186 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:16.186917 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cffcb30-c89d-4fec-864e-d8a46697c186 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:40:16.188354 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cffcb30-c89d-4fec-864e-d8a46697c186 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:40:16.199133 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 754/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:40:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:40:18.248374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35956516-abff-40d9-9a61-96b42d09dd97 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:18.248944 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35956516-abff-40d9-9a61-96b42d09dd97 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:18.248944 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35956516-abff-40d9-9a61-96b42d09dd97 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:18.250342 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 783/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:40:18.255824 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37431ffa-7a35-4727-a45c-c9a360790deb None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:18.256262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37431ffa-7a35-4727-a45c-c9a360790deb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:18.256371 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37431ffa-7a35-4727-a45c-c9a360790deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:18.274691 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-37431ffa-7a35-4727-a45c-c9a360790deb None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:40:18.275884 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-37431ffa-7a35-4727-a45c-c9a360790deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:40:18.283656 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 755/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:40:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:40:18.579148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35956516-abff-40d9-9a61-96b42d09dd97 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:40:18.580842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-35956516-abff-40d9-9a61-96b42d09dd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:40:18.588503 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=84 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:40:18.594834 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35956516-abff-40d9-9a61-96b42d09dd97 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:40:18.598370 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91495376-6bc1-451f-a991-03d6241f4e3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:40:18.599007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91495376-6bc1-451f-a991-03d6241f4e3d service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:40:18.599295 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91495376-6bc1-451f-a991-03d6241f4e3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:40:18.599573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91495376-6bc1-451f-a991-03d6241f4e3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:40:18.609365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91495376-6bc1-451f-a991-03d6241f4e3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:40:18.610840 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91495376-6bc1-451f-a991-03d6241f4e3d service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:40:18.620756 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 784/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:40:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:40:28.270008 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=11 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:40:28.616013 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:47.325246 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ffc2b-f17e-4c15-9da0-7b9b1c1ed881 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.326285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ffc2b-f17e-4c15-9da0-7b9b1c1ed881 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.326739 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ffc2b-f17e-4c15-9da0-7b9b1c1ed881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.353169 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-6c4ffc2b-f17e-4c15-9da0-7b9b1c1ed881 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:47.354717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6c4ffc2b-f17e-4c15-9da0-7b9b1c1ed881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:47.365150 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 756/1540] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:41:47.374134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4d9dd-f267-485f-bc34-42a7e9efd057 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.374729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4d9dd-f267-485f-bc34-42a7e9efd057 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.374827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4d9dd-f267-485f-bc34-42a7e9efd057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.399492 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8ba4d9dd-f267-485f-bc34-42a7e9efd057 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:47.400707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8ba4d9dd-f267-485f-bc34-42a7e9efd057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:47.413110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 785/1541] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:47.419443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ffc2b-f17e-4c15-9da0-7b9b1c1ed881 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:47.431521 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ffc2b-f17e-4c15-9da0-7b9b1c1ed881 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:47.441193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a40c2-8176-4ed5-a076-7b6f8d272adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:47.441877 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a40c2-8176-4ed5-a076-7b6f8d272adf admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.442193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a40c2-8176-4ed5-a076-7b6f8d272adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.442468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a40c2-8176-4ed5-a076-7b6f8d272adf admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.443201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a40c2-8176-4ed5-a076-7b6f8d272adf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:41:47.443506 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a40c2-8176-4ed5-a076-7b6f8d272adf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:47.445154 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a40c2-8176-4ed5-a076-7b6f8d272adf admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:47.454272 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac75661-08b6-462c-8293-6bc2a271e0ec None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.455021 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac75661-08b6-462c-8293-6bc2a271e0ec None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.455021 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac75661-08b6-462c-8293-6bc2a271e0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.458558 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 757/1542] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:41:47.467082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a40c2-8176-4ed5-a076-7b6f8d272adf admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:47.483740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a40c2-8176-4ed5-a076-7b6f8d272adf admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:47.495096 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3ac75661-08b6-462c-8293-6bc2a271e0ec None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:47.496321 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3ac75661-08b6-462c-8293-6bc2a271e0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:47.499081 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06752525-ca62-4b55-80fa-2a20922e438b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:47.499892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06752525-ca62-4b55-80fa-2a20922e438b admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.500177 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06752525-ca62-4b55-80fa-2a20922e438b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.500698 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06752525-ca62-4b55-80fa-2a20922e438b admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.501933 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06752525-ca62-4b55-80fa-2a20922e438b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:47.502997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: WARNING py.warnings [None req-06752525-ca62-4b55-80fa-2a20922e438b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:41:47.502997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: warnings.warn(msg) Jul 12 13:41:47.502997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: Jul 12 13:41:47.504917 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06752525-ca62-4b55-80fa-2a20922e438b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:47.509999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 786/1543] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 12 13:41:47.519235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1d024-c92a-4568-b6ce-5d50c8b94e10 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.519517 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1d024-c92a-4568-b6ce-5d50c8b94e10 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.519733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1d024-c92a-4568-b6ce-5d50c8b94e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.556552 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3bc1d024-c92a-4568-b6ce-5d50c8b94e10 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:47.559944 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3bc1d024-c92a-4568-b6ce-5d50c8b94e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:47.576408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 787/1544] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:41:47.578863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 758/1545] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:47.581236 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1d024-c92a-4568-b6ce-5d50c8b94e10 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:47.583824 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06752525-ca62-4b55-80fa-2a20922e438b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:47.597805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1d024-c92a-4568-b6ce-5d50c8b94e10 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:47.706048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06752525-ca62-4b55-80fa-2a20922e438b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:47.709422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d2683-e2f2-4c5c-bc3c-5aa0ba804fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:47.710115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d2683-e2f2-4c5c-bc3c-5aa0ba804fbd admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.711066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d2683-e2f2-4c5c-bc3c-5aa0ba804fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.711341 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d2683-e2f2-4c5c-bc3c-5aa0ba804fbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.711447 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5cfd0e-9877-4e20-b146-4fb43041d869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:47.712095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d2683-e2f2-4c5c-bc3c-5aa0ba804fbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:41:47.712232 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cfd0e-9877-4e20-b146-4fb43041d869 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.712377 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cfd0e-9877-4e20-b146-4fb43041d869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.712484 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d2683-e2f2-4c5c-bc3c-5aa0ba804fbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:47.712648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cfd0e-9877-4e20-b146-4fb43041d869 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.713349 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5cfd0e-9877-4e20-b146-4fb43041d869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:47.713462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: WARNING py.warnings [None req-411d2683-e2f2-4c5c-bc3c-5aa0ba804fbd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:41:47.713462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: warnings.warn(msg) Jul 12 13:41:47.713462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: Jul 12 13:41:47.714329 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d2683-e2f2-4c5c-bc3c-5aa0ba804fbd admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:47.715296 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5cfd0e-9877-4e20-b146-4fb43041d869 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:47.725991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 788/1546] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:41:47.727202 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f61dfe-9913-4361-8270-0def81a55f81 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.727494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f61dfe-9913-4361-8270-0def81a55f81 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.727746 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f61dfe-9913-4361-8270-0def81a55f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.736161 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 759/1547] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/users => generated 290 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:41:47.739487 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da26b91-c69c-4646-aa8e-1188a439e6a6 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.739780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da26b91-c69c-4646-aa8e-1188a439e6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.740058 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da26b91-c69c-4646-aa8e-1188a439e6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.745686 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d0f61dfe-9913-4361-8270-0def81a55f81 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:47.746906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d0f61dfe-9913-4361-8270-0def81a55f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:47.762711 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 789/1548] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:47.762711 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f65ab-4466-4a13-8450-60ac58e31aff None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.762711 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f65ab-4466-4a13-8450-60ac58e31aff None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.762711 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f65ab-4466-4a13-8450-60ac58e31aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.763124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0da26b91-c69c-4646-aa8e-1188a439e6a6 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:47.763124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0da26b91-c69c-4646-aa8e-1188a439e6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:47.768174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 760/1549] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:47.769355 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa921c0-3c6c-408c-8719-51e7b289d481 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.769646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa921c0-3c6c-408c-8719-51e7b289d481 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.769926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa921c0-3c6c-408c-8719-51e7b289d481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.776783 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-485f65ab-4466-4a13-8450-60ac58e31aff None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:47.777988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-485f65ab-4466-4a13-8450-60ac58e31aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:47.786009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 790/1550] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:47.787374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5392a-b5c6-45e8-ad15-a82366c6dcf9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.787681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5392a-b5c6-45e8-ad15-a82366c6dcf9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.787961 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5392a-b5c6-45e8-ad15-a82366c6dcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.794342 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3aa921c0-3c6c-408c-8719-51e7b289d481 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:47.795560 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3aa921c0-3c6c-408c-8719-51e7b289d481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:47.803959 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 761/1551] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:47.805179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43819443-bac4-4c2a-9472-08b452b2d7a2 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.805468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43819443-bac4-4c2a-9472-08b452b2d7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.805746 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43819443-bac4-4c2a-9472-08b452b2d7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.805864 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-18f5392a-b5c6-45e8-ad15-a82366c6dcf9 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:47.807153 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-18f5392a-b5c6-45e8-ad15-a82366c6dcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:47.815481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 791/1552] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:47.816254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5392a-b5c6-45e8-ad15-a82366c6dcf9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:47.825474 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-43819443-bac4-4c2a-9472-08b452b2d7a2 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:47.826749 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-43819443-bac4-4c2a-9472-08b452b2d7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:47.826914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5392a-b5c6-45e8-ad15-a82366c6dcf9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:47.831458 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e92c9-02c1-4516-81c4-6e0e17d27596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:47.832105 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e92c9-02c1-4516-81c4-6e0e17d27596 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.832390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e92c9-02c1-4516-81c4-6e0e17d27596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.832671 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e92c9-02c1-4516-81c4-6e0e17d27596 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.833353 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e92c9-02c1-4516-81c4-6e0e17d27596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:47.834585 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 762/1553] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:47.835437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43819443-bac4-4c2a-9472-08b452b2d7a2 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:47.835557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e92c9-02c1-4516-81c4-6e0e17d27596 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:47.844850 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43819443-bac4-4c2a-9472-08b452b2d7a2 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:47.849688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c94160-60e8-4613-9d61-b4f648d58307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:47.850376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c94160-60e8-4613-9d61-b4f648d58307 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.850660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c94160-60e8-4613-9d61-b4f648d58307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.851459 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c94160-60e8-4613-9d61-b4f648d58307 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.852181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c94160-60e8-4613-9d61-b4f648d58307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:47.853778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c94160-60e8-4613-9d61-b4f648d58307 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:47.864529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 763/1554] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:47.865824 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a4fba7-e4c6-49d4-bf33-0bca860076b1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.866136 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a4fba7-e4c6-49d4-bf33-0bca860076b1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.866452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a4fba7-e4c6-49d4-bf33-0bca860076b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.867620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 792/1555] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:47.871340 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcc02c-b8e7-4e1e-8986-a2bfeed806fc None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.871637 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcc02c-b8e7-4e1e-8986-a2bfeed806fc None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.871922 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcc02c-b8e7-4e1e-8986-a2bfeed806fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.885338 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-47a4fba7-e4c6-49d4-bf33-0bca860076b1 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:47.886586 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-47a4fba7-e4c6-49d4-bf33-0bca860076b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:47.893121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-78fcc02c-b8e7-4e1e-8986-a2bfeed806fc None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:47.894344 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-78fcc02c-b8e7-4e1e-8986-a2bfeed806fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:47.945207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 764/1556] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:47.946780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2340cce-5e2b-4f18-9fef-82028a2fc98f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.947236 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2340cce-5e2b-4f18-9fef-82028a2fc98f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.947671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2340cce-5e2b-4f18-9fef-82028a2fc98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.953937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 793/1557] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:47.955365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a95ef-67eb-4e37-a985-7caeb211bc9a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.955661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a95ef-67eb-4e37-a985-7caeb211bc9a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.955939 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a95ef-67eb-4e37-a985-7caeb211bc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.969071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d2340cce-5e2b-4f18-9fef-82028a2fc98f None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:47.970370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d2340cce-5e2b-4f18-9fef-82028a2fc98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:47.975147 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-534a95ef-67eb-4e37-a985-7caeb211bc9a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:47.976371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-534a95ef-67eb-4e37-a985-7caeb211bc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:47.983599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 765/1558] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:47.983599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc470096-effa-4b28-958f-9da5500ab668 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.983599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc470096-effa-4b28-958f-9da5500ab668 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.983599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc470096-effa-4b28-958f-9da5500ab668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:47.987655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 794/1559] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:47.988868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:47.989069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c0bbf-f68c-4e51-9ced-747c1cadc42e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:47.989408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c0bbf-f68c-4e51-9ced-747c1cadc42e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:47.989692 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c0bbf-f68c-4e51-9ced-747c1cadc42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.002465 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-cc470096-effa-4b28-958f-9da5500ab668 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:48.003667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cc470096-effa-4b28-958f-9da5500ab668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:48.009136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-037c0bbf-f68c-4e51-9ced-747c1cadc42e None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:48.010341 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-037c0bbf-f68c-4e51-9ced-747c1cadc42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:48.012780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 766/1560] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:48.016737 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc470096-effa-4b28-958f-9da5500ab668 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.018463 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 795/1561] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:48.019224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c0bbf-f68c-4e51-9ced-747c1cadc42e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.023800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc470096-effa-4b28-958f-9da5500ab668 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.028755 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511ce64-4c9d-4b1e-9e33-d7ac15026c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.029415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511ce64-4c9d-4b1e-9e33-d7ac15026c21 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.029697 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511ce64-4c9d-4b1e-9e33-d7ac15026c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.029981 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511ce64-4c9d-4b1e-9e33-d7ac15026c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.030721 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511ce64-4c9d-4b1e-9e33-d7ac15026c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.031415 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c0bbf-f68c-4e51-9ced-747c1cadc42e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.032401 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511ce64-4c9d-4b1e-9e33-d7ac15026c21 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.036195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdaa2ef-db94-4219-8752-e389b788160e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.036843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaa2ef-db94-4219-8752-e389b788160e admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.037126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaa2ef-db94-4219-8752-e389b788160e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.037401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaa2ef-db94-4219-8752-e389b788160e admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.038123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdaa2ef-db94-4219-8752-e389b788160e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.039209 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: WARNING py.warnings [None req-cfdaa2ef-db94-4219-8752-e389b788160e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:41:48.039209 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: warnings.warn(msg) Jul 12 13:41:48.039209 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: Jul 12 13:41:48.040396 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdaa2ef-db94-4219-8752-e389b788160e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.042832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 767/1562] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:48.043519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511ce64-4c9d-4b1e-9e33-d7ac15026c21 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.053361 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511ce64-4c9d-4b1e-9e33-d7ac15026c21 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.058625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9a50c-1aa2-4722-b8c7-e518df46c437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.059300 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9a50c-1aa2-4722-b8c7-e518df46c437 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.059583 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9a50c-1aa2-4722-b8c7-e518df46c437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.059857 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9a50c-1aa2-4722-b8c7-e518df46c437 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.060569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9a50c-1aa2-4722-b8c7-e518df46c437 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:41:48.060881 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9a50c-1aa2-4722-b8c7-e518df46c437 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.064842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 796/1563] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:41:48 2022] POST /identity/v3/users => generated 289 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:48.064842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdaa2ef-db94-4219-8752-e389b788160e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.065157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9a50c-1aa2-4722-b8c7-e518df46c437 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.067259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:48.074160 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdaa2ef-db94-4219-8752-e389b788160e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.082053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 768/1564] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:41:48.082053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9a50c-1aa2-4722-b8c7-e518df46c437 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.085819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9a50c-1aa2-4722-b8c7-e518df46c437 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.086424 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328859f-896f-4c05-a6cd-2f5c5d1eecab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.087116 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328859f-896f-4c05-a6cd-2f5c5d1eecab admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.087399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328859f-896f-4c05-a6cd-2f5c5d1eecab admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.087676 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328859f-896f-4c05-a6cd-2f5c5d1eecab admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.088400 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328859f-896f-4c05-a6cd-2f5c5d1eecab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:41:48.088709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328859f-896f-4c05-a6cd-2f5c5d1eecab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.090361 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328859f-896f-4c05-a6cd-2f5c5d1eecab admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.090486 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4fde4-ba60-440e-a371-6f75f9904f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.091056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4fde4-ba60-440e-a371-6f75f9904f02 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.091338 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4fde4-ba60-440e-a371-6f75f9904f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.091610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4fde4-ba60-440e-a371-6f75f9904f02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.092313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4fde4-ba60-440e-a371-6f75f9904f02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:41:48.092622 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4fde4-ba60-440e-a371-6f75f9904f02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.094203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4fde4-ba60-440e-a371-6f75f9904f02 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.109438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 797/1565] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:41:48.109438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328859f-896f-4c05-a6cd-2f5c5d1eecab admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.109719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 769/1566] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:41:48.109719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4fde4-ba60-440e-a371-6f75f9904f02 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.118572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328859f-896f-4c05-a6cd-2f5c5d1eecab admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.120450 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4fde4-ba60-440e-a371-6f75f9904f02 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.123295 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99c9c5-fdeb-472d-a948-edc1f8d222a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.123947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99c9c5-fdeb-472d-a948-edc1f8d222a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.124234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99c9c5-fdeb-472d-a948-edc1f8d222a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.124512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99c9c5-fdeb-472d-a948-edc1f8d222a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.125020 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e96bd-c449-4cf6-b1cf-a04800964e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.125237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99c9c5-fdeb-472d-a948-edc1f8d222a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:41:48.125548 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99c9c5-fdeb-472d-a948-edc1f8d222a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.125650 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e96bd-c449-4cf6-b1cf-a04800964e64 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.125927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e96bd-c449-4cf6-b1cf-a04800964e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.126477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e96bd-c449-4cf6-b1cf-a04800964e64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.127204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e96bd-c449-4cf6-b1cf-a04800964e64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:41:48.127510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e96bd-c449-4cf6-b1cf-a04800964e64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.128466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99c9c5-fdeb-472d-a948-edc1f8d222a3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.129085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e96bd-c449-4cf6-b1cf-a04800964e64 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.140363 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 798/1567] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:41:48.141274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99c9c5-fdeb-472d-a948-edc1f8d222a3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.141810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 770/1568] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:41:48.142619 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e96bd-c449-4cf6-b1cf-a04800964e64 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.151563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99c9c5-fdeb-472d-a948-edc1f8d222a3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.156493 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8648e3-4c0d-4723-8fbb-1ad833d77a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.157182 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8648e3-4c0d-4723-8fbb-1ad833d77a01 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.157467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8648e3-4c0d-4723-8fbb-1ad833d77a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.157740 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8648e3-4c0d-4723-8fbb-1ad833d77a01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.158483 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8648e3-4c0d-4723-8fbb-1ad833d77a01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:41:48.158791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8648e3-4c0d-4723-8fbb-1ad833d77a01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.160042 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e96bd-c449-4cf6-b1cf-a04800964e64 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.160417 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8648e3-4c0d-4723-8fbb-1ad833d77a01 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.167251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ce924-af50-4f0c-9b70-6116bb53e8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.168822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ce924-af50-4f0c-9b70-6116bb53e8cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.169363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ce924-af50-4f0c-9b70-6116bb53e8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.169878 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ce924-af50-4f0c-9b70-6116bb53e8cd admin admin] PATH_INFO: `/v3/projects/e78f1d11bdee4728a8ed0b39b57c5eab/users/db68bc84a7f94deda6f495d2eb7145ca/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.186279 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 799/1569] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:41:48.186279 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8648e3-4c0d-4723-8fbb-1ad833d77a01 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.188603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ce924-af50-4f0c-9b70-6116bb53e8cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e78f1d11bdee4728a8ed0b39b57c5eab, user_id=db68bc84a7f94deda6f495d2eb7145ca, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.189712 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: WARNING py.warnings [None req-518ce924-af50-4f0c-9b70-6116bb53e8cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:41:48.189712 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: warnings.warn(msg) Jul 12 13:41:48.189712 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: Jul 12 13:41:48.191505 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8648e3-4c0d-4723-8fbb-1ad833d77a01 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.191685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ce924-af50-4f0c-9b70-6116bb53e8cd admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.197315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df2acc-9b30-4e51-ad3a-a183275d34ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.197668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df2acc-9b30-4e51-ad3a-a183275d34ff admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.197948 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df2acc-9b30-4e51-ad3a-a183275d34ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.198321 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df2acc-9b30-4e51-ad3a-a183275d34ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.199030 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df2acc-9b30-4e51-ad3a-a183275d34ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.200620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df2acc-9b30-4e51-ad3a-a183275d34ff admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.206561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 771/1570] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:41:48 2022] PUT /identity/v3/projects/e78f1d11bdee4728a8ed0b39b57c5eab/users/db68bc84a7f94deda6f495d2eb7145ca/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:41:48.207455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ce924-af50-4f0c-9b70-6116bb53e8cd admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.220829 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 800/1571] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:48.221821 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df2acc-9b30-4e51-ad3a-a183275d34ff admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.233142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df2acc-9b30-4e51-ad3a-a183275d34ff admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.269764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ce924-af50-4f0c-9b70-6116bb53e8cd admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.274456 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f1feb-b6bc-463f-a400-ff75f7ea77bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.275114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f1feb-b6bc-463f-a400-ff75f7ea77bd admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.275394 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f1feb-b6bc-463f-a400-ff75f7ea77bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.275665 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f1feb-b6bc-463f-a400-ff75f7ea77bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.276356 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f1feb-b6bc-463f-a400-ff75f7ea77bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.278290 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f1feb-b6bc-463f-a400-ff75f7ea77bd admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.291498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e130b8a-2fd6-48ad-bc58-f303c5e9c902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.292147 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e130b8a-2fd6-48ad-bc58-f303c5e9c902 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.292429 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e130b8a-2fd6-48ad-bc58-f303c5e9c902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.292710 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e130b8a-2fd6-48ad-bc58-f303c5e9c902 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.293394 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e130b8a-2fd6-48ad-bc58-f303c5e9c902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.295362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e130b8a-2fd6-48ad-bc58-f303c5e9c902 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.303390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 772/1572] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:41:48 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:48.304266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f1feb-b6bc-463f-a400-ff75f7ea77bd admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.321547 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 801/1573] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:41:48 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:48.322510 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e130b8a-2fd6-48ad-bc58-f303c5e9c902 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.369077 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f1feb-b6bc-463f-a400-ff75f7ea77bd admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.378150 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:48.385451 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e130b8a-2fd6-48ad-bc58-f303c5e9c902 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.390138 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51410b2c-1b42-4fb7-a81f-05410d5d1994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.390793 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51410b2c-1b42-4fb7-a81f-05410d5d1994 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.391074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51410b2c-1b42-4fb7-a81f-05410d5d1994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.391346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51410b2c-1b42-4fb7-a81f-05410d5d1994 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.392022 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51410b2c-1b42-4fb7-a81f-05410d5d1994 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.393924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51410b2c-1b42-4fb7-a81f-05410d5d1994 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.419705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 802/1574] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:41:48 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:48.420559 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51410b2c-1b42-4fb7-a81f-05410d5d1994 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.432003 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1f2ea-017d-4bab-aa65-fdb435ea396d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.432667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1f2ea-017d-4bab-aa65-fdb435ea396d admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.432949 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1f2ea-017d-4bab-aa65-fdb435ea396d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.433226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1f2ea-017d-4bab-aa65-fdb435ea396d admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.433909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1f2ea-017d-4bab-aa65-fdb435ea396d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.435837 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1f2ea-017d-4bab-aa65-fdb435ea396d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.445234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:48.462609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 773/1575] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:41:48 2022] POST /identity/v3/projects => generated 391 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:48.463545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1f2ea-017d-4bab-aa65-fdb435ea396d admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.486908 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51410b2c-1b42-4fb7-a81f-05410d5d1994 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.524354 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1f2ea-017d-4bab-aa65-fdb435ea396d admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.529031 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee36b18-23fa-42c3-b43e-372844ad30b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.529687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee36b18-23fa-42c3-b43e-372844ad30b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.529987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee36b18-23fa-42c3-b43e-372844ad30b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.530274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee36b18-23fa-42c3-b43e-372844ad30b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.530962 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee36b18-23fa-42c3-b43e-372844ad30b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.533103 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee36b18-23fa-42c3-b43e-372844ad30b3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.544572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39644f65-f250-4765-ac70-e3c965d9dd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.545234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39644f65-f250-4765-ac70-e3c965d9dd4e admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.545521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39644f65-f250-4765-ac70-e3c965d9dd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.545800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39644f65-f250-4765-ac70-e3c965d9dd4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.546518 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39644f65-f250-4765-ac70-e3c965d9dd4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.548629 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39644f65-f250-4765-ac70-e3c965d9dd4e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.559052 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 774/1576] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:41:48 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:48.560336 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c29f0e-35c5-4683-9acb-a30213eabec3 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.560630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c29f0e-35c5-4683-9acb-a30213eabec3 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.560909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c29f0e-35c5-4683-9acb-a30213eabec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.579731 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 803/1577] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:41:48 2022] POST /identity/v3/projects => generated 403 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:48.581036 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39644f65-f250-4765-ac70-e3c965d9dd4e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.591120 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-65c29f0e-35c5-4683-9acb-a30213eabec3 None None] MFA Rules not processed for user `db68bc84a7f94deda6f495d2eb7145ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:48.592318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-65c29f0e-35c5-4683-9acb-a30213eabec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:48.644531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39644f65-f250-4765-ac70-e3c965d9dd4e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.649185 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb706de6-5ebb-45a1-81ea-a8f816ac59fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.649851 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb706de6-5ebb-45a1-81ea-a8f816ac59fd admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.650170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb706de6-5ebb-45a1-81ea-a8f816ac59fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.650452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb706de6-5ebb-45a1-81ea-a8f816ac59fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.651180 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb706de6-5ebb-45a1-81ea-a8f816ac59fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.652794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb706de6-5ebb-45a1-81ea-a8f816ac59fd admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.655932 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 775/1578] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:48.656766 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c29f0e-35c5-4683-9acb-a30213eabec3 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.665231 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 804/1579] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:48.665903 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb706de6-5ebb-45a1-81ea-a8f816ac59fd admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.667289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c29f0e-35c5-4683-9acb-a30213eabec3 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.672263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b0bee-c3f9-49d3-aa4a-eca0b95ddfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.672911 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b0bee-c3f9-49d3-aa4a-eca0b95ddfae admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.673192 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b0bee-c3f9-49d3-aa4a-eca0b95ddfae admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.673467 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b0bee-c3f9-49d3-aa4a-eca0b95ddfae admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.674171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b0bee-c3f9-49d3-aa4a-eca0b95ddfae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.675680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb706de6-5ebb-45a1-81ea-a8f816ac59fd admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.676081 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b0bee-c3f9-49d3-aa4a-eca0b95ddfae admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.680821 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb240e1-fc42-427c-98fc-7fd027c7b31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.681491 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb240e1-fc42-427c-98fc-7fd027c7b31f admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.681780 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb240e1-fc42-427c-98fc-7fd027c7b31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.682097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb240e1-fc42-427c-98fc-7fd027c7b31f admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.682774 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb240e1-fc42-427c-98fc-7fd027c7b31f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.684799 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb240e1-fc42-427c-98fc-7fd027c7b31f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.685716 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:48.701625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 776/1580] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:41:48 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:48.701625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b0bee-c3f9-49d3-aa4a-eca0b95ddfae admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.711182 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 805/1581] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:41:48 2022] POST /identity/v3/users => generated 295 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:48.711182 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb240e1-fc42-427c-98fc-7fd027c7b31f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.713870 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b0bee-c3f9-49d3-aa4a-eca0b95ddfae admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.718868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02af60b-bdde-4caf-bb3d-b6dd9dccbe71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.719095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb240e1-fc42-427c-98fc-7fd027c7b31f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.719519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02af60b-bdde-4caf-bb3d-b6dd9dccbe71 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.719905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02af60b-bdde-4caf-bb3d-b6dd9dccbe71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.720183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02af60b-bdde-4caf-bb3d-b6dd9dccbe71 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.720868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02af60b-bdde-4caf-bb3d-b6dd9dccbe71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.722890 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02af60b-bdde-4caf-bb3d-b6dd9dccbe71 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.724057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d7613-16f0-4278-bbda-e2489658c866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.724780 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d7613-16f0-4278-bbda-e2489658c866 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.725143 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d7613-16f0-4278-bbda-e2489658c866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.725425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d7613-16f0-4278-bbda-e2489658c866 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.726270 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d7613-16f0-4278-bbda-e2489658c866 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.728363 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d7613-16f0-4278-bbda-e2489658c866 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.744049 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 777/1582] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:41:48 2022] POST /identity/v3/users => generated 293 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:48.744912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02af60b-bdde-4caf-bb3d-b6dd9dccbe71 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.758078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 806/1583] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:41:48 2022] POST /identity/v3/users => generated 296 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:48.758292 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02af60b-bdde-4caf-bb3d-b6dd9dccbe71 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.758397 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d7613-16f0-4278-bbda-e2489658c866 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.760089 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b36fb0-6ef7-4dd8-9354-652a68520e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.760836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b36fb0-6ef7-4dd8-9354-652a68520e6f admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.761124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b36fb0-6ef7-4dd8-9354-652a68520e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.761403 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b36fb0-6ef7-4dd8-9354-652a68520e6f admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.762595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b36fb0-6ef7-4dd8-9354-652a68520e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.764041 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b36fb0-6ef7-4dd8-9354-652a68520e6f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.766166 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d7613-16f0-4278-bbda-e2489658c866 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.770728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c70102-ab9e-4753-89a0-5337deecc370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.771375 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c70102-ab9e-4753-89a0-5337deecc370 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.771695 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c70102-ab9e-4753-89a0-5337deecc370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.771971 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c70102-ab9e-4753-89a0-5337deecc370 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.773317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c70102-ab9e-4753-89a0-5337deecc370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.775144 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c70102-ab9e-4753-89a0-5337deecc370 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.787057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 778/1584] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:41:48 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:48.788044 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b36fb0-6ef7-4dd8-9354-652a68520e6f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.793015 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:48.798569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b36fb0-6ef7-4dd8-9354-652a68520e6f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.803945 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 807/1585] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:41:48 2022] POST /identity/v3/users => generated 302 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:48.803945 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c70102-ab9e-4753-89a0-5337deecc370 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.806317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c7305-eb03-4b9d-bba4-3c04f58a889b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.807013 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c7305-eb03-4b9d-bba4-3c04f58a889b admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.807285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c7305-eb03-4b9d-bba4-3c04f58a889b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.807564 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c7305-eb03-4b9d-bba4-3c04f58a889b admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.808298 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c7305-eb03-4b9d-bba4-3c04f58a889b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.810274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c7305-eb03-4b9d-bba4-3c04f58a889b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.815124 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c70102-ab9e-4753-89a0-5337deecc370 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.820141 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea437b72-9c35-4366-9b00-afed10bb384e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.821024 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea437b72-9c35-4366-9b00-afed10bb384e admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.821304 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea437b72-9c35-4366-9b00-afed10bb384e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.821577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea437b72-9c35-4366-9b00-afed10bb384e admin admin] PATH_INFO: `/v3/projects/3f5464e5ca174b398ebb28ad3e8a84f1/users/c25bdc429c3c4f8ba33ee4042f128c8e/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.837103 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 779/1586] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:41:48 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:48.837666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea437b72-9c35-4366-9b00-afed10bb384e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f5464e5ca174b398ebb28ad3e8a84f1, user_id=c25bdc429c3c4f8ba33ee4042f128c8e, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.838317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c7305-eb03-4b9d-bba4-3c04f58a889b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.844182 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea437b72-9c35-4366-9b00-afed10bb384e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.855787 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 808/1587] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:41:48 2022] PUT /identity/v3/projects/3f5464e5ca174b398ebb28ad3e8a84f1/users/c25bdc429c3c4f8ba33ee4042f128c8e/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:41:48.856604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea437b72-9c35-4366-9b00-afed10bb384e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.909422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c7305-eb03-4b9d-bba4-3c04f58a889b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.919715 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea437b72-9c35-4366-9b00-afed10bb384e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.924459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3103ae-9a77-469b-8cc6-58849912ad86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.925119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3103ae-9a77-469b-8cc6-58849912ad86 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.925412 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3103ae-9a77-469b-8cc6-58849912ad86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.925666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3103ae-9a77-469b-8cc6-58849912ad86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.926457 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3103ae-9a77-469b-8cc6-58849912ad86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.928229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3103ae-9a77-469b-8cc6-58849912ad86 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.938571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 809/1588] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:48.942346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3103ae-9a77-469b-8cc6-58849912ad86 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.951927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3103ae-9a77-469b-8cc6-58849912ad86 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.956530 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf6d8c-54da-4759-9772-44619b8896c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.957170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf6d8c-54da-4759-9772-44619b8896c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.957461 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf6d8c-54da-4759-9772-44619b8896c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.957737 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf6d8c-54da-4759-9772-44619b8896c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.958499 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf6d8c-54da-4759-9772-44619b8896c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.960673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf6d8c-54da-4759-9772-44619b8896c3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.967029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4e9d0-d6cc-4fa6-bddf-41e064f628e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.967695 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4e9d0-d6cc-4fa6-bddf-41e064f628e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.967974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4e9d0-d6cc-4fa6-bddf-41e064f628e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.968247 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4e9d0-d6cc-4fa6-bddf-41e064f628e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.968965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4e9d0-d6cc-4fa6-bddf-41e064f628e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.970317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 810/1589] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:48.970569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4e9d0-d6cc-4fa6-bddf-41e064f628e5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.971098 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf6d8c-54da-4759-9772-44619b8896c3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.981021 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf6d8c-54da-4759-9772-44619b8896c3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.984345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 780/1590] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:48.984345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4e9d0-d6cc-4fa6-bddf-41e064f628e5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:48.988732 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731571f-59dd-48ba-a307-4de0f037698e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.989370 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731571f-59dd-48ba-a307-4de0f037698e admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.989656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731571f-59dd-48ba-a307-4de0f037698e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.989934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731571f-59dd-48ba-a307-4de0f037698e admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.990720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731571f-59dd-48ba-a307-4de0f037698e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:48.992036 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4e9d0-d6cc-4fa6-bddf-41e064f628e5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:48.992341 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731571f-59dd-48ba-a307-4de0f037698e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:48.996524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a3cff-726d-4bb9-b286-2cebc9cf2ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:48.997154 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a3cff-726d-4bb9-b286-2cebc9cf2ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:48.997436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a3cff-726d-4bb9-b286-2cebc9cf2ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:48.997707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a3cff-726d-4bb9-b286-2cebc9cf2ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:48.998516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a3cff-726d-4bb9-b286-2cebc9cf2ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.000073 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a3cff-726d-4bb9-b286-2cebc9cf2ca9 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.003093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 811/1591] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:49.003872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731571f-59dd-48ba-a307-4de0f037698e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.013370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 781/1592] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:49.013370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a3cff-726d-4bb9-b286-2cebc9cf2ca9 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.013632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731571f-59dd-48ba-a307-4de0f037698e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.018244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35458b-bf23-47cf-b569-203b7fe3f420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.018890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35458b-bf23-47cf-b569-203b7fe3f420 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.019175 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35458b-bf23-47cf-b569-203b7fe3f420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.019458 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35458b-bf23-47cf-b569-203b7fe3f420 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.020169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35458b-bf23-47cf-b569-203b7fe3f420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.021766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35458b-bf23-47cf-b569-203b7fe3f420 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.024132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a3cff-726d-4bb9-b286-2cebc9cf2ca9 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.029559 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ef620-ed0c-4b64-b0d7-9a20a241b03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.030302 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ef620-ed0c-4b64-b0d7-9a20a241b03b admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.030302 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ef620-ed0c-4b64-b0d7-9a20a241b03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.031273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ef620-ed0c-4b64-b0d7-9a20a241b03b admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.032495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 812/1593] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:49.032657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ef620-ed0c-4b64-b0d7-9a20a241b03b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.033701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3d191-8fb8-46c3-80a5-639776c906e0 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.034002 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3d191-8fb8-46c3-80a5-639776c906e0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.034279 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3d191-8fb8-46c3-80a5-639776c906e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.034542 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ef620-ed0c-4b64-b0d7-9a20a241b03b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.052152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-48e3d191-8fb8-46c3-80a5-639776c906e0 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:49.053360 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-48e3d191-8fb8-46c3-80a5-639776c906e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:49.055437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 782/1594] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:41:49 2022] POST /identity/v3/users => generated 297 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:49.055437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ef620-ed0c-4b64-b0d7-9a20a241b03b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.064838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 813/1595] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:49.064838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3d191-8fb8-46c3-80a5-639776c906e0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.071148 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ef620-ed0c-4b64-b0d7-9a20a241b03b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.072638 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3d191-8fb8-46c3-80a5-639776c906e0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.075941 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ef512-ce03-4d05-a4b5-50eee6493d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.076596 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ef512-ce03-4d05-a4b5-50eee6493d2a admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.076899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ef512-ce03-4d05-a4b5-50eee6493d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.077177 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ef512-ce03-4d05-a4b5-50eee6493d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.077742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244bdab0-a69e-462f-8f0f-d4636f5f0840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.077897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ef512-ce03-4d05-a4b5-50eee6493d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.078402 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bdab0-a69e-462f-8f0f-d4636f5f0840 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.078687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bdab0-a69e-462f-8f0f-d4636f5f0840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.078963 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bdab0-a69e-462f-8f0f-d4636f5f0840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.079534 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ef512-ce03-4d05-a4b5-50eee6493d2a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.079674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244bdab0-a69e-462f-8f0f-d4636f5f0840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.081260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244bdab0-a69e-462f-8f0f-d4636f5f0840 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.090760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 783/1596] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:49.091619 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ef512-ce03-4d05-a4b5-50eee6493d2a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.094249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 814/1597] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:49.094249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244bdab0-a69e-462f-8f0f-d4636f5f0840 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.097647 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:49.103721 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244bdab0-a69e-462f-8f0f-d4636f5f0840 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.104362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ef512-ce03-4d05-a4b5-50eee6493d2a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.108510 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496aa94f-d62c-4d5e-b335-70c1f1e992cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.109416 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496aa94f-d62c-4d5e-b335-70c1f1e992cf admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.109602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaf883-3f89-46c4-b42f-1e5fa60557e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.109732 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496aa94f-d62c-4d5e-b335-70c1f1e992cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.109972 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496aa94f-d62c-4d5e-b335-70c1f1e992cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.110338 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaf883-3f89-46c4-b42f-1e5fa60557e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.110625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaf883-3f89-46c4-b42f-1e5fa60557e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.110738 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496aa94f-d62c-4d5e-b335-70c1f1e992cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.110914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaf883-3f89-46c4-b42f-1e5fa60557e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.111681 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aaf883-3f89-46c4-b42f-1e5fa60557e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.112260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496aa94f-d62c-4d5e-b335-70c1f1e992cf admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.113271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aaf883-3f89-46c4-b42f-1e5fa60557e5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.123004 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 815/1598] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:49.126889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 784/1599] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:49.127303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496aa94f-d62c-4d5e-b335-70c1f1e992cf admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.127455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaf883-3f89-46c4-b42f-1e5fa60557e5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.134983 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496aa94f-d62c-4d5e-b335-70c1f1e992cf admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.136651 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaf883-3f89-46c4-b42f-1e5fa60557e5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.139500 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef365f-8309-4582-b991-f580274a9e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.140145 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef365f-8309-4582-b991-f580274a9e96 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.140429 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef365f-8309-4582-b991-f580274a9e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.140705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef365f-8309-4582-b991-f580274a9e96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.141414 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef365f-8309-4582-b991-f580274a9e96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.141810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90977db8-67e8-4b50-9851-acbe8a7d1227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.142482 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90977db8-67e8-4b50-9851-acbe8a7d1227 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.142765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90977db8-67e8-4b50-9851-acbe8a7d1227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.143037 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef365f-8309-4582-b991-f580274a9e96 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.143155 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90977db8-67e8-4b50-9851-acbe8a7d1227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.143774 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90977db8-67e8-4b50-9851-acbe8a7d1227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.145348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90977db8-67e8-4b50-9851-acbe8a7d1227 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.153192 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 816/1600] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:49.154095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef365f-8309-4582-b991-f580274a9e96 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.158403 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 785/1601] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:49.158403 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90977db8-67e8-4b50-9851-acbe8a7d1227 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.164467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef365f-8309-4582-b991-f580274a9e96 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.168980 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ebf2f-1abe-4840-ab24-fb3b3e8745fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.169611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ebf2f-1abe-4840-ab24-fb3b3e8745fb admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.169894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ebf2f-1abe-4840-ab24-fb3b3e8745fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.170209 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ebf2f-1abe-4840-ab24-fb3b3e8745fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.170922 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ebf2f-1abe-4840-ab24-fb3b3e8745fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.172493 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ebf2f-1abe-4840-ab24-fb3b3e8745fb admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.182523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 817/1602] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:49.183315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ebf2f-1abe-4840-ab24-fb3b3e8745fb admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.193157 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ebf2f-1abe-4840-ab24-fb3b3e8745fb admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.198276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7eb7d5-d13d-40b7-b4ce-66ea18e3e4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.199130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7eb7d5-d13d-40b7-b4ce-66ea18e3e4f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.199419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7eb7d5-d13d-40b7-b4ce-66ea18e3e4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.199732 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7eb7d5-d13d-40b7-b4ce-66ea18e3e4f8 admin admin] PATH_INFO: `/v3/projects/e54281ff11484f77b0d20fd039ef938e/users/d9e5bc258de54fd6b3811401783a7408/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.213383 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90977db8-67e8-4b50-9851-acbe8a7d1227 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.214150 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7eb7d5-d13d-40b7-b4ce-66ea18e3e4f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e54281ff11484f77b0d20fd039ef938e, user_id=d9e5bc258de54fd6b3811401783a7408, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.216798 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7eb7d5-d13d-40b7-b4ce-66ea18e3e4f8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.217156 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107feca-491d-442c-9f4a-9a20d3d38ca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.217796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107feca-491d-442c-9f4a-9a20d3d38ca4 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.218116 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107feca-491d-442c-9f4a-9a20d3d38ca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.218397 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107feca-491d-442c-9f4a-9a20d3d38ca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.228918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107feca-491d-442c-9f4a-9a20d3d38ca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.230395 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107feca-491d-442c-9f4a-9a20d3d38ca4 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.231338 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 818/1603] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:41:49 2022] PUT /identity/v3/projects/e54281ff11484f77b0d20fd039ef938e/users/d9e5bc258de54fd6b3811401783a7408/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:41:49.232146 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7eb7d5-d13d-40b7-b4ce-66ea18e3e4f8 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.239964 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 786/1604] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:41:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:41:49.240738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107feca-491d-442c-9f4a-9a20d3d38ca4 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.249696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107feca-491d-442c-9f4a-9a20d3d38ca4 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.258631 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:49.300226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7eb7d5-d13d-40b7-b4ce-66ea18e3e4f8 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.305007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503cf725-ce81-4a3c-89d7-d9676e1aaa46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.305907 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503cf725-ce81-4a3c-89d7-d9676e1aaa46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.306230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503cf725-ce81-4a3c-89d7-d9676e1aaa46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.306540 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503cf725-ce81-4a3c-89d7-d9676e1aaa46 admin admin] PATH_INFO: `/v3/projects/c4e88457b140443093476f9b6d9bcff4/users/a892b2511463404c818c63e870a23389/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.318181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6fef8-f754-496c-82e0-362ea2525fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.319147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6fef8-f754-496c-82e0-362ea2525fcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.320097 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6fef8-f754-496c-82e0-362ea2525fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.320441 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6fef8-f754-496c-82e0-362ea2525fcd admin admin] PATH_INFO: `/v3/projects/026f214eb4ee4a9c807b928710c93db5/users/4a6b3bd40ad744908a05df7410cace52/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.324698 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503cf725-ce81-4a3c-89d7-d9676e1aaa46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e88457b140443093476f9b6d9bcff4, user_id=a892b2511463404c818c63e870a23389, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.328044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503cf725-ce81-4a3c-89d7-d9676e1aaa46 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.338324 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6fef8-f754-496c-82e0-362ea2525fcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=026f214eb4ee4a9c807b928710c93db5, user_id=4a6b3bd40ad744908a05df7410cace52, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.341237 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6fef8-f754-496c-82e0-362ea2525fcd admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.343743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 819/1605] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:41:49 2022] PUT /identity/v3/projects/c4e88457b140443093476f9b6d9bcff4/users/a892b2511463404c818c63e870a23389/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:41:49.344294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503cf725-ce81-4a3c-89d7-d9676e1aaa46 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.358842 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 787/1606] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:41:49 2022] PUT /identity/v3/projects/026f214eb4ee4a9c807b928710c93db5/users/4a6b3bd40ad744908a05df7410cace52/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:41:49.359646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6fef8-f754-496c-82e0-362ea2525fcd admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.417555 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503cf725-ce81-4a3c-89d7-d9676e1aaa46 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.466791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6fef8-f754-496c-82e0-362ea2525fcd admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.476396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8960ae2-6c65-47ea-8e6a-79f7da877012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.476975 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8960ae2-6c65-47ea-8e6a-79f7da877012 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.477369 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8960ae2-6c65-47ea-8e6a-79f7da877012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.477847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8960ae2-6c65-47ea-8e6a-79f7da877012 admin admin] PATH_INFO: `/v3/projects/39665fbd2b3e4900814a6b38511daa92/users/7b9f6f3742e84dfb92bed76ea8ea38fb/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.490507 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:49.495432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8960ae2-6c65-47ea-8e6a-79f7da877012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39665fbd2b3e4900814a6b38511daa92, user_id=7b9f6f3742e84dfb92bed76ea8ea38fb, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.498618 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8960ae2-6c65-47ea-8e6a-79f7da877012 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.506817 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed230823-a6c0-414d-9ad3-75a1d89baa4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.508312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed230823-a6c0-414d-9ad3-75a1d89baa4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.508632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed230823-a6c0-414d-9ad3-75a1d89baa4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.508867 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed230823-a6c0-414d-9ad3-75a1d89baa4f admin admin] PATH_INFO: `/v3/projects/5f431e3ab50a4133916f6175a2ef1b4f/users/2ad8ebe54a1d42eda9b23108f92a1d23/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.516729 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 788/1607] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:41:49 2022] PUT /identity/v3/projects/39665fbd2b3e4900814a6b38511daa92/users/7b9f6f3742e84dfb92bed76ea8ea38fb/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:41:49.517830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8960ae2-6c65-47ea-8e6a-79f7da877012 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.529196 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed230823-a6c0-414d-9ad3-75a1d89baa4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f431e3ab50a4133916f6175a2ef1b4f, user_id=2ad8ebe54a1d42eda9b23108f92a1d23, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.531218 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed230823-a6c0-414d-9ad3-75a1d89baa4f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.548715 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 820/1608] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:41:49 2022] PUT /identity/v3/projects/5f431e3ab50a4133916f6175a2ef1b4f/users/2ad8ebe54a1d42eda9b23108f92a1d23/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:41:49.549072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed230823-a6c0-414d-9ad3-75a1d89baa4f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.603395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:49.605048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8960ae2-6c65-47ea-8e6a-79f7da877012 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.635591 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed230823-a6c0-414d-9ad3-75a1d89baa4f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.642171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe83ad-a9a9-4a9a-8bcc-28ad734d3b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.643197 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe83ad-a9a9-4a9a-8bcc-28ad734d3b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.643636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe83ad-a9a9-4a9a-8bcc-28ad734d3b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.643913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe83ad-a9a9-4a9a-8bcc-28ad734d3b51 admin admin] PATH_INFO: `/v3/projects/f73a4312030644ea86622ffd31cb2355/users/8f62a70154ee47a7964c2bbb6cc65ee8/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.662760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe83ad-a9a9-4a9a-8bcc-28ad734d3b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f73a4312030644ea86622ffd31cb2355, user_id=8f62a70154ee47a7964c2bbb6cc65ee8, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.666243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe83ad-a9a9-4a9a-8bcc-28ad734d3b51 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.675936 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd0513-ed54-4344-abf1-01c07825cf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.677061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd0513-ed54-4344-abf1-01c07825cf07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.677344 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd0513-ed54-4344-abf1-01c07825cf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.677661 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd0513-ed54-4344-abf1-01c07825cf07 admin admin] PATH_INFO: `/v3/projects/404b12bef8ea4d788b1e352576911b8c/users/12826723aeef4f2785a12beb27dbcb31/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.685588 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 821/1609] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:41:49 2022] PUT /identity/v3/projects/f73a4312030644ea86622ffd31cb2355/users/8f62a70154ee47a7964c2bbb6cc65ee8/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:41:49.688437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b643dc-d37c-4f67-8744-30d17c8899fe None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.688437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b643dc-d37c-4f67-8744-30d17c8899fe None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.688437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b643dc-d37c-4f67-8744-30d17c8899fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.698700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd0513-ed54-4344-abf1-01c07825cf07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=404b12bef8ea4d788b1e352576911b8c, user_id=12826723aeef4f2785a12beb27dbcb31, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.701519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd0513-ed54-4344-abf1-01c07825cf07 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.707080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-56b643dc-d37c-4f67-8744-30d17c8899fe None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:49.708412 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-56b643dc-d37c-4f67-8744-30d17c8899fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:49.717920 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 789/1610] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:41:49 2022] PUT /identity/v3/projects/404b12bef8ea4d788b1e352576911b8c/users/12826723aeef4f2785a12beb27dbcb31/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:41:49.719429 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1f0cc-1bc3-45e1-ad58-928a7e6a8d2e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.719822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1f0cc-1bc3-45e1-ad58-928a7e6a8d2e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.720203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1f0cc-1bc3-45e1-ad58-928a7e6a8d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.740231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-cac1f0cc-1bc3-45e1-ad58-928a7e6a8d2e None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:49.742213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cac1f0cc-1bc3-45e1-ad58-928a7e6a8d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:49.780179 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 822/1611] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:49.782164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add63148-df63-46fb-82c6-fce5e94d3c96 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.782732 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add63148-df63-46fb-82c6-fce5e94d3c96 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.782937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add63148-df63-46fb-82c6-fce5e94d3c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.805974 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-add63148-df63-46fb-82c6-fce5e94d3c96 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:49.807366 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-add63148-df63-46fb-82c6-fce5e94d3c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:49.810095 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 790/1612] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:49.812439 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6a1ec-0112-4e93-826c-853996a63de7 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.812780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6a1ec-0112-4e93-826c-853996a63de7 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.813069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6a1ec-0112-4e93-826c-853996a63de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.819644 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 823/1613] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:49.821258 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d492fc-52c4-47fe-ac4a-e1ad9a178e3f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.821641 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d492fc-52c4-47fe-ac4a-e1ad9a178e3f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.822048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d492fc-52c4-47fe-ac4a-e1ad9a178e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.840766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-18d492fc-52c4-47fe-ac4a-e1ad9a178e3f None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:49.842146 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-18d492fc-52c4-47fe-ac4a-e1ad9a178e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:49.846747 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-bcb6a1ec-0112-4e93-826c-853996a63de7 None None] MFA Rules not processed for user `7b9f6f3742e84dfb92bed76ea8ea38fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:49.847950 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bcb6a1ec-0112-4e93-826c-853996a63de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:49.862217 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:49.866213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 824/1614] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:49.867780 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d492fc-52c4-47fe-ac4a-e1ad9a178e3f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.878227 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d492fc-52c4-47fe-ac4a-e1ad9a178e3f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:49.883727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6514720-1d44-46ae-b804-5de76a08c3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:49.884468 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6514720-1d44-46ae-b804-5de76a08c3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.884825 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6514720-1d44-46ae-b804-5de76a08c3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.885175 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6514720-1d44-46ae-b804-5de76a08c3a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.886024 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6514720-1d44-46ae-b804-5de76a08c3a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:49.887741 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6514720-1d44-46ae-b804-5de76a08c3a8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:49.899843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 825/1615] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:49.901559 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2f75a-b6ed-454a-81a4-6eaf4394f81f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:49.901863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2f75a-b6ed-454a-81a4-6eaf4394f81f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:49.902177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2f75a-b6ed-454a-81a4-6eaf4394f81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:49.922975 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-2cc2f75a-b6ed-454a-81a4-6eaf4394f81f None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:49.924194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2cc2f75a-b6ed-454a-81a4-6eaf4394f81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:49.937279 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 826/1616] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:49.937279 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2f75a-b6ed-454a-81a4-6eaf4394f81f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:49.952909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 791/1617] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:49.954726 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6a1ec-0112-4e93-826c-853996a63de7 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:50.016542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2f75a-b6ed-454a-81a4-6eaf4394f81f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:50.022617 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82acbd4-7155-4514-8cab-40e5be75e3cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:50.023363 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82acbd4-7155-4514-8cab-40e5be75e3cb service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:50.023650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82acbd4-7155-4514-8cab-40e5be75e3cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:50.023929 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82acbd4-7155-4514-8cab-40e5be75e3cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:50.040452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82acbd4-7155-4514-8cab-40e5be75e3cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:50.042096 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82acbd4-7155-4514-8cab-40e5be75e3cb service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:50.044008 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6a1ec-0112-4e93-826c-853996a63de7 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:50.052600 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a07f63-01a5-444a-8656-3c84b2b652bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:50.053521 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a07f63-01a5-444a-8656-3c84b2b652bb service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:50.053845 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a07f63-01a5-444a-8656-3c84b2b652bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:50.054903 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a07f63-01a5-444a-8656-3c84b2b652bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:50.058047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 827/1618] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:41:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:41:50.058992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82acbd4-7155-4514-8cab-40e5be75e3cb service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:50.065684 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a07f63-01a5-444a-8656-3c84b2b652bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:50.067397 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a07f63-01a5-444a-8656-3c84b2b652bb service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:50.073302 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82acbd4-7155-4514-8cab-40e5be75e3cb service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:50.078859 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70aa60-7f22-4bde-aa48-64bb80262a36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:50.079576 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70aa60-7f22-4bde-aa48-64bb80262a36 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:50.080422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70aa60-7f22-4bde-aa48-64bb80262a36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:50.080709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70aa60-7f22-4bde-aa48-64bb80262a36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:50.089046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 792/1619] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:41:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:41:50.089786 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a07f63-01a5-444a-8656-3c84b2b652bb service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:50.093840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-74a07f63-01a5-444a-8656-3c84b2b652bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:50.101760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c70aa60-7f22-4bde-aa48-64bb80262a36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:50.105064 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c70aa60-7f22-4bde-aa48-64bb80262a36 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:50.122951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a07f63-01a5-444a-8656-3c84b2b652bb service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:50.134083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 828/1620] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:41:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:41:50.134934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70aa60-7f22-4bde-aa48-64bb80262a36 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:50.135932 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132a931-5e68-41f0-9ec1-9f8c9dea2d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:50.136595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132a931-5e68-41f0-9ec1-9f8c9dea2d19 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:50.136879 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132a931-5e68-41f0-9ec1-9f8c9dea2d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:50.137154 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132a931-5e68-41f0-9ec1-9f8c9dea2d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:50.153309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70aa60-7f22-4bde-aa48-64bb80262a36 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:50.158928 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132a931-5e68-41f0-9ec1-9f8c9dea2d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:50.160264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132a931-5e68-41f0-9ec1-9f8c9dea2d19 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:50.169899 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0864fb7-724c-4e51-8f19-ed17011c9369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:50.172831 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0864fb7-724c-4e51-8f19-ed17011c9369 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:50.173129 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0864fb7-724c-4e51-8f19-ed17011c9369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:50.174277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 793/1621] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:41:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:41:50.175004 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0864fb7-724c-4e51-8f19-ed17011c9369 admin admin] PATH_INFO: `/v3/projects/f73a4312030644ea86622ffd31cb2355/users/8f62a70154ee47a7964c2bbb6cc65ee8/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:50.175138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132a931-5e68-41f0-9ec1-9f8c9dea2d19 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:50.181814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0864fb7-724c-4e51-8f19-ed17011c9369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f73a4312030644ea86622ffd31cb2355, user_id=8f62a70154ee47a7964c2bbb6cc65ee8, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:50.188591 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132a931-5e68-41f0-9ec1-9f8c9dea2d19 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:50.191331 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0864fb7-724c-4e51-8f19-ed17011c9369 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:50.194332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a8ee5-651a-4d50-838d-e45e8c47c27b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:50.195453 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a8ee5-651a-4d50-838d-e45e8c47c27b service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:50.196095 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a8ee5-651a-4d50-838d-e45e8c47c27b service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:50.196789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a8ee5-651a-4d50-838d-e45e8c47c27b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:50.214026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a8ee5-651a-4d50-838d-e45e8c47c27b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:50.217065 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 829/1622] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:41:50 2022] PUT /identity/v3/projects/f73a4312030644ea86622ffd31cb2355/users/8f62a70154ee47a7964c2bbb6cc65ee8/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:41:50.218316 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a8ee5-651a-4d50-838d-e45e8c47c27b service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:50.220578 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9c716-8484-41be-9005-af4dd3e84708 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:50.222215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9c716-8484-41be-9005-af4dd3e84708 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:50.222215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9c716-8484-41be-9005-af4dd3e84708 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:50.226705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 830/1623] 10.222.0.52 () {54 vars in 961 bytes} [Tue Jul 12 13:41:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:41:50.230279 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b151d7-a53a-4dd2-9a20-94163ea05219 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:50.231330 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b151d7-a53a-4dd2-9a20-94163ea05219 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:50.231733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b151d7-a53a-4dd2-9a20-94163ea05219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:50.244576 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 794/1624] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:41:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:41:50.246424 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce163be-b3de-4f16-8ec1-bcd7b5fbed54 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:50.248014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce163be-b3de-4f16-8ec1-bcd7b5fbed54 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:50.248315 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce163be-b3de-4f16-8ec1-bcd7b5fbed54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:50.248496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:50.295645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b8b151d7-a53a-4dd2-9a20-94163ea05219 None None] MFA Rules not processed for user `8f62a70154ee47a7964c2bbb6cc65ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:50.298292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b8b151d7-a53a-4dd2-9a20-94163ea05219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:50.331566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-5ce163be-b3de-4f16-8ec1-bcd7b5fbed54 None None] MFA Rules not processed for user `0d3915cde0d04bcfb9b5ce8c89af1e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:50.335872 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5ce163be-b3de-4f16-8ec1-bcd7b5fbed54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:50.461407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 831/1625] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:50.474206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b151d7-a53a-4dd2-9a20-94163ea05219 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:50.488967 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 795/1626] 10.222.0.52 () {58 vars in 1058 bytes} [Tue Jul 12 13:41:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:50.498448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce163be-b3de-4f16-8ec1-bcd7b5fbed54 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:50.510787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:50.521781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce163be-b3de-4f16-8ec1-bcd7b5fbed54 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:50.541606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ba5e1-63a3-4c30-9b5f-090fadf90fcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:50.542309 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ba5e1-63a3-4c30-9b5f-090fadf90fcf service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:50.543901 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ba5e1-63a3-4c30-9b5f-090fadf90fcf service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:50.546202 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ba5e1-63a3-4c30-9b5f-090fadf90fcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:50.598525 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b151d7-a53a-4dd2-9a20-94163ea05219 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:50.610497 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a1bba-61b9-4eab-a5e1-c923f1503d88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:50.613025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a1bba-61b9-4eab-a5e1-c923f1503d88 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:50.613025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a1bba-61b9-4eab-a5e1-c923f1503d88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:50.613025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a1bba-61b9-4eab-a5e1-c923f1503d88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:50.633332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a1bba-61b9-4eab-a5e1-c923f1503d88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:50.635926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a1bba-61b9-4eab-a5e1-c923f1503d88 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:50.688242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ba5e1-63a3-4c30-9b5f-090fadf90fcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:50.691060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 832/1627] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:41:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:41:50.695702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ba5e1-63a3-4c30-9b5f-090fadf90fcf service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:50.736331 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 796/1628] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:41:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:41:50.863006 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a1bba-61b9-4eab-a5e1-c923f1503d88 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:50.878815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a1bba-61b9-4eab-a5e1-c923f1503d88 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:50.883526 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f6d28-cd2a-4c8b-86c5-adb0c79efff2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:50.884151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f6d28-cd2a-4c8b-86c5-adb0c79efff2 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:50.887193 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f6d28-cd2a-4c8b-86c5-adb0c79efff2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:50.887495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f6d28-cd2a-4c8b-86c5-adb0c79efff2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:51.027301 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f6d28-cd2a-4c8b-86c5-adb0c79efff2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:51.028896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f6d28-cd2a-4c8b-86c5-adb0c79efff2 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:51.065179 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 833/1629] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:41:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:41:51.097625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba75a4-fe08-4c4c-893b-16d14175be0f None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:51.097918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba75a4-fe08-4c4c-893b-16d14175be0f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:51.098351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba75a4-fe08-4c4c-893b-16d14175be0f None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:51.100065 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 797/1630] 10.222.0.52 () {54 vars in 912 bytes} [Tue Jul 12 13:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:41:51.106654 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074b174-ed4d-49a4-bb98-e6699a57fbf1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:51.107033 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074b174-ed4d-49a4-bb98-e6699a57fbf1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:51.107448 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074b174-ed4d-49a4-bb98-e6699a57fbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:51.139117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c074b174-ed4d-49a4-bb98-e6699a57fbf1 None None] MFA Rules not processed for user `0d3915cde0d04bcfb9b5ce8c89af1e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:51.142236 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c074b174-ed4d-49a4-bb98-e6699a57fbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:51.172512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 834/1631] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jul 12 13:41:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:51.180152 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba75a4-fe08-4c4c-893b-16d14175be0f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:51.198423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba75a4-fe08-4c4c-893b-16d14175be0f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:51.210922 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0475ee-abb8-4679-912f-eafacff3bfad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:51.211700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0475ee-abb8-4679-912f-eafacff3bfad service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:51.212507 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0475ee-abb8-4679-912f-eafacff3bfad service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:51.213285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0475ee-abb8-4679-912f-eafacff3bfad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:51.232827 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0475ee-abb8-4679-912f-eafacff3bfad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:51.235036 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0475ee-abb8-4679-912f-eafacff3bfad service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:51.319996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 798/1632] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:41:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:41:52.702439 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731f076-cd30-421e-b194-480f68c80c3f None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:52.702948 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731f076-cd30-421e-b194-480f68c80c3f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:52.703195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731f076-cd30-421e-b194-480f68c80c3f None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:52.705843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 835/1633] 10.222.0.52 () {54 vars in 961 bytes} [Tue Jul 12 13:41:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:41:52.723024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96468498-ce9d-4e57-9b67-276b68ae0292 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:52.723546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96468498-ce9d-4e57-9b67-276b68ae0292 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:52.723660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96468498-ce9d-4e57-9b67-276b68ae0292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:52.766402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-96468498-ce9d-4e57-9b67-276b68ae0292 None None] MFA Rules not processed for user `0d3915cde0d04bcfb9b5ce8c89af1e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:52.767786 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-96468498-ce9d-4e57-9b67-276b68ae0292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:52.795363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 799/1634] 10.222.0.52 () {58 vars in 1058 bytes} [Tue Jul 12 13:41:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:52.800977 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7461e-8bc8-4c46-a6b7-b8da3d72a512 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:52.801524 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7461e-8bc8-4c46-a6b7-b8da3d72a512 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:52.802015 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7461e-8bc8-4c46-a6b7-b8da3d72a512 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:52.803708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 836/1635] 10.222.0.52 () {54 vars in 912 bytes} [Tue Jul 12 13:41:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:41:52.810578 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ccfc27-cb70-4ab7-8e0d-77ce3327bb4d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:52.811130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ccfc27-cb70-4ab7-8e0d-77ce3327bb4d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:52.811642 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ccfc27-cb70-4ab7-8e0d-77ce3327bb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:52.862510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d2ccfc27-cb70-4ab7-8e0d-77ce3327bb4d None None] MFA Rules not processed for user `0d3915cde0d04bcfb9b5ce8c89af1e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:52.864729 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d2ccfc27-cb70-4ab7-8e0d-77ce3327bb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:52.880104 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 800/1636] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jul 12 13:41:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:52.894283 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7461e-8bc8-4c46-a6b7-b8da3d72a512 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:52.911647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7461e-8bc8-4c46-a6b7-b8da3d72a512 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:52.923898 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85e0a4-4744-4cf0-81ee-a8bf004a306a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:52.924660 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85e0a4-4744-4cf0-81ee-a8bf004a306a service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:52.925636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85e0a4-4744-4cf0-81ee-a8bf004a306a service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:52.926274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85e0a4-4744-4cf0-81ee-a8bf004a306a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:52.956695 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae85e0a4-4744-4cf0-81ee-a8bf004a306a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:52.960305 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae85e0a4-4744-4cf0-81ee-a8bf004a306a service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:53.007489 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 837/1637] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:41:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:41:53.378602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d2955-2d39-4ddd-b49e-ecf81d0fa63a None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:53.379168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d2955-2d39-4ddd-b49e-ecf81d0fa63a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:53.379605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d2955-2d39-4ddd-b49e-ecf81d0fa63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:53.464222 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8a5d2955-2d39-4ddd-b49e-ecf81d0fa63a None None] MFA Rules not processed for user `12826723aeef4f2785a12beb27dbcb31`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:53.465548 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8a5d2955-2d39-4ddd-b49e-ecf81d0fa63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:53.599100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 801/1638] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:53.603208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85e0a4-4744-4cf0-81ee-a8bf004a306a service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:53.626033 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:53.763892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85e0a4-4744-4cf0-81ee-a8bf004a306a service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:53.778416 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b83a8b-416c-475c-9cad-37fca9b24fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:53.779736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b83a8b-416c-475c-9cad-37fca9b24fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:53.780028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b83a8b-416c-475c-9cad-37fca9b24fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:53.780441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b83a8b-416c-475c-9cad-37fca9b24fd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:53.781148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b83a8b-416c-475c-9cad-37fca9b24fd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:53.783142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b83a8b-416c-475c-9cad-37fca9b24fd8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:53.856968 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 838/1639] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:41:53 2022] POST /identity/v3/projects => generated 405 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:53.861275 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d2955-2d39-4ddd-b49e-ecf81d0fa63a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:53.887553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d2955-2d39-4ddd-b49e-ecf81d0fa63a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:54.067830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af8e1b-aa7a-4a5c-952a-b76a92e35d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:54.068616 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af8e1b-aa7a-4a5c-952a-b76a92e35d9f admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:54.070541 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af8e1b-aa7a-4a5c-952a-b76a92e35d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:54.070949 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af8e1b-aa7a-4a5c-952a-b76a92e35d9f admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:54.071652 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af8e1b-aa7a-4a5c-952a-b76a92e35d9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:54.073630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af8e1b-aa7a-4a5c-952a-b76a92e35d9f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:54.099508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:54.124764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 802/1640] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:41:53 2022] POST /identity/v3/users => generated 309 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:54.128930 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b83a8b-416c-475c-9cad-37fca9b24fd8 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:54.157096 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b83a8b-416c-475c-9cad-37fca9b24fd8 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:54.172012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf9f7b-aadb-44eb-9d0c-bd4b2cd2b8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:54.172982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf9f7b-aadb-44eb-9d0c-bd4b2cd2b8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:54.173449 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf9f7b-aadb-44eb-9d0c-bd4b2cd2b8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:54.173880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf9f7b-aadb-44eb-9d0c-bd4b2cd2b8f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:54.174830 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf9f7b-aadb-44eb-9d0c-bd4b2cd2b8f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:54.184884 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf9f7b-aadb-44eb-9d0c-bd4b2cd2b8f0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:54.205454 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 839/1641] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:54 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:54.210764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af8e1b-aa7a-4a5c-952a-b76a92e35d9f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:54.233905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af8e1b-aa7a-4a5c-952a-b76a92e35d9f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:54.249018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b70bb-c42f-4406-8713-70211413cff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:54.251575 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b70bb-c42f-4406-8713-70211413cff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:54.252396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b70bb-c42f-4406-8713-70211413cff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:54.253935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b70bb-c42f-4406-8713-70211413cff2 admin admin] PATH_INFO: `/v3/projects/f8a7610659d64826ad0fc963ec83e9ca/users/a6b0d2cb54424479a6229441834dbfe8/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:54.308620 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b70bb-c42f-4406-8713-70211413cff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8a7610659d64826ad0fc963ec83e9ca, user_id=a6b0d2cb54424479a6229441834dbfe8, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:54.318411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b70bb-c42f-4406-8713-70211413cff2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:54.362438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 803/1642] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:41:54 2022] PUT /identity/v3/projects/f8a7610659d64826ad0fc963ec83e9ca/users/a6b0d2cb54424479a6229441834dbfe8/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:41:54.365905 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf9f7b-aadb-44eb-9d0c-bd4b2cd2b8f0 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:54.376827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf9f7b-aadb-44eb-9d0c-bd4b2cd2b8f0 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:54.518790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c3a71-37d3-4caa-b968-5a0ba2cb84b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:54.519496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c3a71-37d3-4caa-b968-5a0ba2cb84b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:54.519783 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c3a71-37d3-4caa-b968-5a0ba2cb84b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:54.520059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c3a71-37d3-4caa-b968-5a0ba2cb84b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:54.520796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c3a71-37d3-4caa-b968-5a0ba2cb84b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:54.523144 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c3a71-37d3-4caa-b968-5a0ba2cb84b9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:54.541035 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 840/1643] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:54 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:54.542645 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b70bb-c42f-4406-8713-70211413cff2 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:54.561708 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b70bb-c42f-4406-8713-70211413cff2 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:54.578432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6e8c8-3fba-44a7-adf8-2edb96f4a32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:54.581307 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6e8c8-3fba-44a7-adf8-2edb96f4a32e admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:54.589944 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6e8c8-3fba-44a7-adf8-2edb96f4a32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:54.597297 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6e8c8-3fba-44a7-adf8-2edb96f4a32e admin admin] PATH_INFO: `/v3/projects/f8a7610659d64826ad0fc963ec83e9ca/users/a6b0d2cb54424479a6229441834dbfe8/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:54.609417 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6e8c8-3fba-44a7-adf8-2edb96f4a32e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8a7610659d64826ad0fc963ec83e9ca, user_id=a6b0d2cb54424479a6229441834dbfe8, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:54.621000 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6e8c8-3fba-44a7-adf8-2edb96f4a32e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:54.655143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 804/1644] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:41:54 2022] PUT /identity/v3/projects/f8a7610659d64826ad0fc963ec83e9ca/users/a6b0d2cb54424479a6229441834dbfe8/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:41:54.894904 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c3a71-37d3-4caa-b968-5a0ba2cb84b9 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:54.910181 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c3a71-37d3-4caa-b968-5a0ba2cb84b9 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:55.051695 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a26e7f-3e4b-47b6-a270-91615918ba1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:55.052872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a26e7f-3e4b-47b6-a270-91615918ba1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:55.053745 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a26e7f-3e4b-47b6-a270-91615918ba1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:55.058421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a26e7f-3e4b-47b6-a270-91615918ba1b admin admin] PATH_INFO: `/v3/users/7b9f6f3742e84dfb92bed76ea8ea38fb` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:55.064883 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a26e7f-3e4b-47b6-a270-91615918ba1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b9f6f3742e84dfb92bed76ea8ea38fb)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:55.067147 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a26e7f-3e4b-47b6-a270-91615918ba1b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:55.141990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:55.210255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c1a26e7f-3e4b-47b6-a270-91615918ba1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b9f6f3742e84dfb92bed76ea8ea38fb'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:41:55.212252 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c1a26e7f-3e4b-47b6-a270-91615918ba1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b9f6f3742e84dfb92bed76ea8ea38fb'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:41:55.268556 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c1a26e7f-3e4b-47b6-a270-91615918ba1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b9f6f3742e84dfb92bed76ea8ea38fb'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:41:55.268556 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c1a26e7f-3e4b-47b6-a270-91615918ba1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b9f6f3742e84dfb92bed76ea8ea38fb'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:41:55.311766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c1a26e7f-3e4b-47b6-a270-91615918ba1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b9f6f3742e84dfb92bed76ea8ea38fb'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:41:55.352141 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 841/1645] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:41:54 2022] DELETE /identity/v3/users/7b9f6f3742e84dfb92bed76ea8ea38fb => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:41:55.356702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b39709-d4dc-4719-8c4b-9b6f5334aaac None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:55.357268 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b39709-d4dc-4719-8c4b-9b6f5334aaac None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:55.357367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b39709-d4dc-4719-8c4b-9b6f5334aaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:55.405306 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-66b39709-d4dc-4719-8c4b-9b6f5334aaac None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:55.406536 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-66b39709-d4dc-4719-8c4b-9b6f5334aaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:55.547023 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 805/1646] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:55.560596 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a26e7f-3e4b-47b6-a270-91615918ba1b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:55.565382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c1a26e7f-3e4b-47b6-a270-91615918ba1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:55.681800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a26e7f-3e4b-47b6-a270-91615918ba1b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:55.688628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c3e0f-8056-4f02-b841-b5148951ce75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:55.689546 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c3e0f-8056-4f02-b841-b5148951ce75 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:55.690290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c3e0f-8056-4f02-b841-b5148951ce75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:55.691167 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c3e0f-8056-4f02-b841-b5148951ce75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:55.719196 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c3e0f-8056-4f02-b841-b5148951ce75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:55.721227 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c3e0f-8056-4f02-b841-b5148951ce75 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:55.736464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 842/1647] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:41:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:41:57.833947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90f8f6-46af-4d4b-ba9f-6d37bb6f9e1c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:57.837872 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90f8f6-46af-4d4b-ba9f-6d37bb6f9e1c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:57.838362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90f8f6-46af-4d4b-ba9f-6d37bb6f9e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:57.932264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-6a90f8f6-46af-4d4b-ba9f-6d37bb6f9e1c None None] MFA Rules not processed for user `a6b0d2cb54424479a6229441834dbfe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:57.933348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6a90f8f6-46af-4d4b-ba9f-6d37bb6f9e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:58.098487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c3e0f-8056-4f02-b841-b5148951ce75 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:58.101167 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e69c3e0f-8056-4f02-b841-b5148951ce75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:58.140796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c3e0f-8056-4f02-b841-b5148951ce75 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:58.159035 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbc9b4-e307-4748-a33a-c53d4a15d675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:58.165039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbc9b4-e307-4748-a33a-c53d4a15d675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:58.167118 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbc9b4-e307-4748-a33a-c53d4a15d675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:58.167651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbc9b4-e307-4748-a33a-c53d4a15d675 admin admin] PATH_INFO: `/v3/projects/39665fbd2b3e4900814a6b38511daa92` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:58.177481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbc9b4-e307-4748-a33a-c53d4a15d675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39665fbd2b3e4900814a6b38511daa92)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:58.177481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: WARNING py.warnings [None req-d9bbc9b4-e307-4748-a33a-c53d4a15d675 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:41:58.177481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: warnings.warn(msg) Jul 12 13:41:58.177481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: Jul 12 13:41:58.178688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 806/1648] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:57 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:58.181741 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbc9b4-e307-4748-a33a-c53d4a15d675 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:58.306591 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d9bbc9b4-e307-4748-a33a-c53d4a15d675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39665fbd2b3e4900814a6b38511daa92', 'request_id': 'req-d9bbc9b4-e307-4748-a33a-c53d4a15d675'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:41:58.354079 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d9bbc9b4-e307-4748-a33a-c53d4a15d675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39665fbd2b3e4900814a6b38511daa92', 'request_id': 'req-d9bbc9b4-e307-4748-a33a-c53d4a15d675'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:41:58.400323 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d9bbc9b4-e307-4748-a33a-c53d4a15d675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39665fbd2b3e4900814a6b38511daa92', 'request_id': 'req-d9bbc9b4-e307-4748-a33a-c53d4a15d675'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:41:58.456723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d9bbc9b4-e307-4748-a33a-c53d4a15d675 admin admin] The token cache is being invalidate because project 39665fbd2b3e4900814a6b38511daa92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:41:58.456723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d9bbc9b4-e307-4748-a33a-c53d4a15d675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:41:58.458306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d9bbc9b4-e307-4748-a33a-c53d4a15d675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:41:58.466823 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 843/1649] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:41:58 2022] DELETE /identity/v3/projects/39665fbd2b3e4900814a6b38511daa92 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:41:58.472527 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17a4cd-fbcb-4d50-b099-6b31c53ed6ee None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:58.473148 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17a4cd-fbcb-4d50-b099-6b31c53ed6ee None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:58.473628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17a4cd-fbcb-4d50-b099-6b31c53ed6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:58.520784 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ac17a4cd-fbcb-4d50-b099-6b31c53ed6ee None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:58.522935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ac17a4cd-fbcb-4d50-b099-6b31c53ed6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:58.679828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:58.740188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 807/1650] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:58.748538 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dd6c7-a30a-4f87-8043-eeff6b638b88 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:58.748538 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dd6c7-a30a-4f87-8043-eeff6b638b88 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:58.748538 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dd6c7-a30a-4f87-8043-eeff6b638b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:58.794706 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ef5dd6c7-a30a-4f87-8043-eeff6b638b88 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:58.796958 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ef5dd6c7-a30a-4f87-8043-eeff6b638b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:58.817833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 844/1651] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:41:58.822590 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17a4cd-fbcb-4d50-b099-6b31c53ed6ee None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:58.837067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17a4cd-fbcb-4d50-b099-6b31c53ed6ee None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:58.866971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a16c8-6870-4502-86e2-d9cbda29fa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:58.867765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a16c8-6870-4502-86e2-d9cbda29fa49 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:58.868056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a16c8-6870-4502-86e2-d9cbda29fa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:58.868335 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a16c8-6870-4502-86e2-d9cbda29fa49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:58.869279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a16c8-6870-4502-86e2-d9cbda29fa49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:41:58.869736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a16c8-6870-4502-86e2-d9cbda29fa49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:58.871532 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a16c8-6870-4502-86e2-d9cbda29fa49 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:58.901583 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 808/1652] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:41:58.906615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5dd6c7-a30a-4f87-8043-eeff6b638b88 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:58.922814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5dd6c7-a30a-4f87-8043-eeff6b638b88 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:58.933722 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb3d8e-3229-4cc8-a6e3-b2e2cab8a673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:58.934933 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb3d8e-3229-4cc8-a6e3-b2e2cab8a673 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:58.935407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb3d8e-3229-4cc8-a6e3-b2e2cab8a673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:58.935875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb3d8e-3229-4cc8-a6e3-b2e2cab8a673 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:58.936780 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb3d8e-3229-4cc8-a6e3-b2e2cab8a673 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:58.939057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb3d8e-3229-4cc8-a6e3-b2e2cab8a673 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:58.990454 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:59.011482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 845/1653] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:41:58 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:59.015440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a16c8-6870-4502-86e2-d9cbda29fa49 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:59.029090 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a16c8-6870-4502-86e2-d9cbda29fa49 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:59.127575 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917c63e-251b-4f6e-ac56-f51feaabecb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:59.128786 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917c63e-251b-4f6e-ac56-f51feaabecb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:59.129663 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917c63e-251b-4f6e-ac56-f51feaabecb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:59.130900 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917c63e-251b-4f6e-ac56-f51feaabecb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:59.134795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917c63e-251b-4f6e-ac56-f51feaabecb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:59.141985 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917c63e-251b-4f6e-ac56-f51feaabecb1 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:59.177683 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 809/1654] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:41:59 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:59.181702 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb3d8e-3229-4cc8-a6e3-b2e2cab8a673 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:59.199265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb3d8e-3229-4cc8-a6e3-b2e2cab8a673 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:59.205256 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5295dec-60c4-4532-bf88-da6a2e445b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:59.206487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5295dec-60c4-4532-bf88-da6a2e445b86 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:59.206977 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5295dec-60c4-4532-bf88-da6a2e445b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:59.207452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5295dec-60c4-4532-bf88-da6a2e445b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:59.208798 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5295dec-60c4-4532-bf88-da6a2e445b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:59.212077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5295dec-60c4-4532-bf88-da6a2e445b86 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:59.233580 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 846/1655] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:41:59.237302 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917c63e-251b-4f6e-ac56-f51feaabecb1 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:59.266287 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917c63e-251b-4f6e-ac56-f51feaabecb1 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:59.280175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb8d6f-dd2a-4d36-8040-d497148e0827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:59.280896 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb8d6f-dd2a-4d36-8040-d497148e0827 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:59.281161 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb8d6f-dd2a-4d36-8040-d497148e0827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:59.281515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb8d6f-dd2a-4d36-8040-d497148e0827 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:59.282197 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb8d6f-dd2a-4d36-8040-d497148e0827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:59.283827 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb8d6f-dd2a-4d36-8040-d497148e0827 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:59.321506 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5295dec-60c4-4532-bf88-da6a2e445b86 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:59.322357 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 810/1656] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:41:59.345760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5295dec-60c4-4532-bf88-da6a2e445b86 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:59.353015 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b31ec-720e-4d63-8b1f-394f9d52efcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:59.360078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b31ec-720e-4d63-8b1f-394f9d52efcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:59.360719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b31ec-720e-4d63-8b1f-394f9d52efcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:59.365920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b31ec-720e-4d63-8b1f-394f9d52efcd admin admin] PATH_INFO: `/v3/projects/9e4da71d5a8f4c7c8aa9a4457f984a53/users/2f7c0ccd5cea4dfb80d041d44c835a5e/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:59.423811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b31ec-720e-4d63-8b1f-394f9d52efcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e4da71d5a8f4c7c8aa9a4457f984a53, user_id=2f7c0ccd5cea4dfb80d041d44c835a5e, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:59.430536 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b31ec-720e-4d63-8b1f-394f9d52efcd admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:59.473338 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca58b10-89fe-4abb-879d-96eddd69cb32 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:59.473338 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca58b10-89fe-4abb-879d-96eddd69cb32 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:59.473338 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca58b10-89fe-4abb-879d-96eddd69cb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:59.474195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 847/1657] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:41:59 2022] PUT /identity/v3/projects/9e4da71d5a8f4c7c8aa9a4457f984a53/users/2f7c0ccd5cea4dfb80d041d44c835a5e/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:41:59.521504 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-dca58b10-89fe-4abb-879d-96eddd69cb32 None None] MFA Rules not processed for user `2f7c0ccd5cea4dfb80d041d44c835a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:41:59.523122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-dca58b10-89fe-4abb-879d-96eddd69cb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:59.700668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 811/1658] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:41:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:41:59.702124 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b31ec-720e-4d63-8b1f-394f9d52efcd admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:59.839598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b31ec-720e-4d63-8b1f-394f9d52efcd admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:59.859716 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e1c47-dbf1-436f-824c-f4cdecb76cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:59.860792 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e1c47-dbf1-436f-824c-f4cdecb76cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:59.861319 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e1c47-dbf1-436f-824c-f4cdecb76cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:59.861833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e1c47-dbf1-436f-824c-f4cdecb76cb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:59.862901 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e1c47-dbf1-436f-824c-f4cdecb76cb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:59.865205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e1c47-dbf1-436f-824c-f4cdecb76cb3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:59.940636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 848/1659] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:41:59 2022] POST /identity/v3/projects => generated 373 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:41:59.944861 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca58b10-89fe-4abb-879d-96eddd69cb32 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:59.963471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca58b10-89fe-4abb-879d-96eddd69cb32 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:00.082267 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:00.123689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694ea69-d8a0-46d1-ad09-187594e22267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:00.124535 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694ea69-d8a0-46d1-ad09-187594e22267 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:00.124829 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694ea69-d8a0-46d1-ad09-187594e22267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:00.125106 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694ea69-d8a0-46d1-ad09-187594e22267 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:00.125813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694ea69-d8a0-46d1-ad09-187594e22267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:00.128093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694ea69-d8a0-46d1-ad09-187594e22267 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:00.174569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 812/1660] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:41:59 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:00.178602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e1c47-dbf1-436f-824c-f4cdecb76cb3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:00.195961 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e1c47-dbf1-436f-824c-f4cdecb76cb3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:00.203773 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc3617-fbe8-4453-a007-ee5ce87863c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:00.204468 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc3617-fbe8-4453-a007-ee5ce87863c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:00.204757 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc3617-fbe8-4453-a007-ee5ce87863c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:00.205035 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc3617-fbe8-4453-a007-ee5ce87863c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:00.205837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc3617-fbe8-4453-a007-ee5ce87863c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:00.208518 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc3617-fbe8-4453-a007-ee5ce87863c8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:00.246706 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 849/1661] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:00.250778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694ea69-d8a0-46d1-ad09-187594e22267 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:00.266458 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694ea69-d8a0-46d1-ad09-187594e22267 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:00.281410 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718a97c-95c7-426b-bd39-cb50dd2e96db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:00.282819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718a97c-95c7-426b-bd39-cb50dd2e96db admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:00.283284 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718a97c-95c7-426b-bd39-cb50dd2e96db admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:00.283737 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718a97c-95c7-426b-bd39-cb50dd2e96db admin admin] PATH_INFO: `/v3/projects/c6fe68a410124081bd4aec3cd35c3751/users/edb18b1a5e14429dbafb543e3d3bb617/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:00.319872 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718a97c-95c7-426b-bd39-cb50dd2e96db admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6fe68a410124081bd4aec3cd35c3751, user_id=edb18b1a5e14429dbafb543e3d3bb617, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:00.322924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718a97c-95c7-426b-bd39-cb50dd2e96db admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:00.356701 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 813/1662] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:42:00 2022] PUT /identity/v3/projects/c6fe68a410124081bd4aec3cd35c3751/users/edb18b1a5e14429dbafb543e3d3bb617/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:00.360874 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc3617-fbe8-4453-a007-ee5ce87863c8 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:00.376658 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc3617-fbe8-4453-a007-ee5ce87863c8 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:00.536187 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce987eb-c4bd-4acc-9c60-f58d65cd9902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:00.537018 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce987eb-c4bd-4acc-9c60-f58d65cd9902 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:00.537746 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce987eb-c4bd-4acc-9c60-f58d65cd9902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:00.538201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce987eb-c4bd-4acc-9c60-f58d65cd9902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:00.539363 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce987eb-c4bd-4acc-9c60-f58d65cd9902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:00.543634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce987eb-c4bd-4acc-9c60-f58d65cd9902 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:00.574510 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 850/1663] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:00.578618 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718a97c-95c7-426b-bd39-cb50dd2e96db admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:00.604669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718a97c-95c7-426b-bd39-cb50dd2e96db admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:00.611637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45ac98-dd62-4b16-9db5-1e34fca6bdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:00.612787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45ac98-dd62-4b16-9db5-1e34fca6bdf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:00.613266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45ac98-dd62-4b16-9db5-1e34fca6bdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:00.613783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45ac98-dd62-4b16-9db5-1e34fca6bdf7 admin admin] PATH_INFO: `/v3/projects/c6fe68a410124081bd4aec3cd35c3751/users/edb18b1a5e14429dbafb543e3d3bb617/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:00.616270 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45ac98-dd62-4b16-9db5-1e34fca6bdf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6fe68a410124081bd4aec3cd35c3751, user_id=edb18b1a5e14429dbafb543e3d3bb617, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:00.619562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45ac98-dd62-4b16-9db5-1e34fca6bdf7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:00.659257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 814/1664] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:42:00 2022] PUT /identity/v3/projects/c6fe68a410124081bd4aec3cd35c3751/users/edb18b1a5e14429dbafb543e3d3bb617/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:00.674722 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb311a9-85d2-4bea-bd3e-f01023283bdc None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:00.679597 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb311a9-85d2-4bea-bd3e-f01023283bdc None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:00.679597 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb311a9-85d2-4bea-bd3e-f01023283bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:00.749767 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5bb311a9-85d2-4bea-bd3e-f01023283bdc None None] MFA Rules not processed for user `edb18b1a5e14429dbafb543e3d3bb617`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:00.752612 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5bb311a9-85d2-4bea-bd3e-f01023283bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:00.788731 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:00.858809 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 851/1665] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:00.872795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45ac98-dd62-4b16-9db5-1e34fca6bdf7 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:00.874648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fe45ac98-dd62-4b16-9db5-1e34fca6bdf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:01.120915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45ac98-dd62-4b16-9db5-1e34fca6bdf7 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:01.141348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ded44-24d1-4b3c-b0be-13ef4bc88752 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:01.142168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ded44-24d1-4b3c-b0be-13ef4bc88752 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:01.142359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ded44-24d1-4b3c-b0be-13ef4bc88752 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:01.142642 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ded44-24d1-4b3c-b0be-13ef4bc88752 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:01.167909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ded44-24d1-4b3c-b0be-13ef4bc88752 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:01.169396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ded44-24d1-4b3c-b0be-13ef4bc88752 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:01.197761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 815/1666] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:01.320894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb311a9-85d2-4bea-bd3e-f01023283bdc None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:01.332670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb311a9-85d2-4bea-bd3e-f01023283bdc None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:01.345914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e72b1d-6a4a-4ca9-a44d-b72f747d3947 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:01.346686 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e72b1d-6a4a-4ca9-a44d-b72f747d3947 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:01.346976 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e72b1d-6a4a-4ca9-a44d-b72f747d3947 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:01.347254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e72b1d-6a4a-4ca9-a44d-b72f747d3947 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:01.528197 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e72b1d-6a4a-4ca9-a44d-b72f747d3947 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:01.529837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e72b1d-6a4a-4ca9-a44d-b72f747d3947 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:01.551355 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 852/1667] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:02.338358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6080b94-7e36-4b63-8b3f-a4b1df90e4e1 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:02.339270 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6080b94-7e36-4b63-8b3f-a4b1df90e4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:02.340792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6080b94-7e36-4b63-8b3f-a4b1df90e4e1 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:02.344449 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 816/1668] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jul 12 13:42:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:42:02.389939 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5780d-ced4-4f3d-8fdc-83cdb02330a5 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:02.390416 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5780d-ced4-4f3d-8fdc-83cdb02330a5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:02.390657 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5780d-ced4-4f3d-8fdc-83cdb02330a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:02.429488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5bc5780d-ced4-4f3d-8fdc-83cdb02330a5 None None] MFA Rules not processed for user `f1308deea1ea4629a1ac22d4b903ad4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:02.430839 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5bc5780d-ced4-4f3d-8fdc-83cdb02330a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:02.595533 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 853/1669] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jul 12 13:42:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:02.645773 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec721d5-289b-479e-9791-b682c5f0bddd None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:02.646021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec721d5-289b-479e-9791-b682c5f0bddd None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:02.646712 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec721d5-289b-479e-9791-b682c5f0bddd None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:02.648951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 817/1670] 10.222.0.52 () {54 vars in 960 bytes} [Tue Jul 12 13:42:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:42:02.660459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a126e4-858f-4d16-985a-f4bb2d6cafe1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:02.661091 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a126e4-858f-4d16-985a-f4bb2d6cafe1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:02.661856 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a126e4-858f-4d16-985a-f4bb2d6cafe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:02.702434 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-75a126e4-858f-4d16-985a-f4bb2d6cafe1 None None] MFA Rules not processed for user `f1308deea1ea4629a1ac22d4b903ad4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:02.703701 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-75a126e4-858f-4d16-985a-f4bb2d6cafe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:02.719799 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 854/1671] 10.222.0.52 () {58 vars in 1057 bytes} [Tue Jul 12 13:42:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:42:02.728736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec721d5-289b-479e-9791-b682c5f0bddd None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:02.759122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec721d5-289b-479e-9791-b682c5f0bddd None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:02.786864 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea7583-fdb3-4da4-8816-ee07a7397faa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1308deea1ea4629a1ac22d4b903ad4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:02.788466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea7583-fdb3-4da4-8816-ee07a7397faa service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:02.788937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea7583-fdb3-4da4-8816-ee07a7397faa service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:02.790839 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea7583-fdb3-4da4-8816-ee07a7397faa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:02.824572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea7583-fdb3-4da4-8816-ee07a7397faa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:02.827469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea7583-fdb3-4da4-8816-ee07a7397faa service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:02.904595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 818/1672] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:02.979502 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6b558-bfec-4d43-ae9b-fe5a9dcfbb84 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:02.980055 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6b558-bfec-4d43-ae9b-fe5a9dcfbb84 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:02.980680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6b558-bfec-4d43-ae9b-fe5a9dcfbb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:03.045165 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-13f6b558-bfec-4d43-ae9b-fe5a9dcfbb84 None None] MFA Rules not processed for user `c25bdc429c3c4f8ba33ee4042f128c8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:03.046885 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-13f6b558-bfec-4d43-ae9b-fe5a9dcfbb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:03.152368 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:03.221648 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 855/1673] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:03.233442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9bca1-4d1b-454f-85ec-9403c47541da None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:03.233711 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9bca1-4d1b-454f-85ec-9403c47541da None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:03.235886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9bca1-4d1b-454f-85ec-9403c47541da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:03.302375 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-c5c9bca1-4d1b-454f-85ec-9403c47541da None None] MFA Rules not processed for user `c25bdc429c3c4f8ba33ee4042f128c8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:03.309384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c5c9bca1-4d1b-454f-85ec-9403c47541da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:03.329780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 819/1674] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:03.343202 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6b558-bfec-4d43-ae9b-fe5a9dcfbb84 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:03.344850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-13f6b558-bfec-4d43-ae9b-fe5a9dcfbb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:03.411798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bdbd2-5be0-40ad-b2b2-36d02c8ec86a None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:03.412260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bdbd2-5be0-40ad-b2b2-36d02c8ec86a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:03.412329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bdbd2-5be0-40ad-b2b2-36d02c8ec86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:03.493170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-440bdbd2-5be0-40ad-b2b2-36d02c8ec86a None None] MFA Rules not processed for user `4a6b3bd40ad744908a05df7410cace52`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:03.495588 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6b558-bfec-4d43-ae9b-fe5a9dcfbb84 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:03.496657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-440bdbd2-5be0-40ad-b2b2-36d02c8ec86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:03.507863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8948e-c223-4ef2-bec9-30a31f089033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:03.508553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8948e-c223-4ef2-bec9-30a31f089033 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:03.508893 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8948e-c223-4ef2-bec9-30a31f089033 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:03.509228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8948e-c223-4ef2-bec9-30a31f089033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:03.521411 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8948e-c223-4ef2-bec9-30a31f089033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:03.526990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8948e-c223-4ef2-bec9-30a31f089033 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:03.582775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 856/1675] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:03.732871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 820/1676] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:42:03.743763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd7c25-5541-40af-9e50-ed7145ee8701 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:03.744234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd7c25-5541-40af-9e50-ed7145ee8701 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:03.744650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd7c25-5541-40af-9e50-ed7145ee8701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:03.799038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-20cd7c25-5541-40af-9e50-ed7145ee8701 None None] MFA Rules not processed for user `4a6b3bd40ad744908a05df7410cace52`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:03.801247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-20cd7c25-5541-40af-9e50-ed7145ee8701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:03.804128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440bdbd2-5be0-40ad-b2b2-36d02c8ec86a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:03.805976 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-440bdbd2-5be0-40ad-b2b2-36d02c8ec86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:03.811390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 857/1677] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:03.817329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:03.828223 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd7c25-5541-40af-9e50-ed7145ee8701 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:03.831442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-20cd7c25-5541-40af-9e50-ed7145ee8701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:03.872710 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd7c25-5541-40af-9e50-ed7145ee8701 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:03.888790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b8034-3387-4341-b09a-b68fcf170cfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:03.889468 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b8034-3387-4341-b09a-b68fcf170cfc service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:03.890869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b8034-3387-4341-b09a-b68fcf170cfc service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:03.891167 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b8034-3387-4341-b09a-b68fcf170cfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:03.913567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b8034-3387-4341-b09a-b68fcf170cfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:03.914840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440bdbd2-5be0-40ad-b2b2-36d02c8ec86a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:03.918410 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b8034-3387-4341-b09a-b68fcf170cfc service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:03.920311 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407400ee-2f09-466a-89a3-12a16a316306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:03.921487 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407400ee-2f09-466a-89a3-12a16a316306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:03.921995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407400ee-2f09-466a-89a3-12a16a316306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:03.922497 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407400ee-2f09-466a-89a3-12a16a316306 admin admin] PATH_INFO: `/v3/users/db68bc84a7f94deda6f495d2eb7145ca` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:03.923976 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407400ee-2f09-466a-89a3-12a16a316306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db68bc84a7f94deda6f495d2eb7145ca)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:03.925218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: WARNING py.warnings [None req-407400ee-2f09-466a-89a3-12a16a316306 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:42:03.925218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: warnings.warn(msg) Jul 12 13:42:03.925218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: Jul 12 13:42:03.926809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407400ee-2f09-466a-89a3-12a16a316306 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:03.942632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 858/1678] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:04.033795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-407400ee-2f09-466a-89a3-12a16a316306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db68bc84a7f94deda6f495d2eb7145ca'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:04.089257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-407400ee-2f09-466a-89a3-12a16a316306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db68bc84a7f94deda6f495d2eb7145ca'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:04.125300 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-407400ee-2f09-466a-89a3-12a16a316306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db68bc84a7f94deda6f495d2eb7145ca'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:04.127129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-407400ee-2f09-466a-89a3-12a16a316306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db68bc84a7f94deda6f495d2eb7145ca'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:04.149676 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-407400ee-2f09-466a-89a3-12a16a316306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db68bc84a7f94deda6f495d2eb7145ca'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:04.159025 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 821/1679] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:42:03 2022] DELETE /identity/v3/users/db68bc84a7f94deda6f495d2eb7145ca => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:04.162901 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e86cb-09f0-452a-b267-5b7e273e7a34 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:04.163200 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e86cb-09f0-452a-b267-5b7e273e7a34 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:04.163484 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e86cb-09f0-452a-b267-5b7e273e7a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:04.190888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3a2e86cb-09f0-452a-b267-5b7e273e7a34 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:04.192218 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3a2e86cb-09f0-452a-b267-5b7e273e7a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:04.322837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 859/1680] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:42:04.357550 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407400ee-2f09-466a-89a3-12a16a316306 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:04.359698 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-407400ee-2f09-466a-89a3-12a16a316306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:04.500744 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407400ee-2f09-466a-89a3-12a16a316306 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:04.514816 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220d9fb-8b5f-4d81-9ed5-186b19a0391c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:04.515506 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220d9fb-8b5f-4d81-9ed5-186b19a0391c service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:04.515790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220d9fb-8b5f-4d81-9ed5-186b19a0391c service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:04.516063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220d9fb-8b5f-4d81-9ed5-186b19a0391c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:04.543741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220d9fb-8b5f-4d81-9ed5-186b19a0391c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:04.545688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220d9fb-8b5f-4d81-9ed5-186b19a0391c service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:04.569176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 822/1681] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:05.096370 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff7670-f025-4f1b-82d8-039723b180b6 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:05.096823 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff7670-f025-4f1b-82d8-039723b180b6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:05.096963 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff7670-f025-4f1b-82d8-039723b180b6 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:05.098862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 860/1682] 10.222.0.52 () {54 vars in 912 bytes} [Tue Jul 12 13:42:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:42:05.104732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a64d5-9311-4483-b89f-55fbc016bbb8 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:05.106753 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a64d5-9311-4483-b89f-55fbc016bbb8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:05.107203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a64d5-9311-4483-b89f-55fbc016bbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:05.236234 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a70a64d5-9311-4483-b89f-55fbc016bbb8 None None] MFA Rules not processed for user `3e833722fe4e493e9ddc4b275632536f`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:05.238933 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a70a64d5-9311-4483-b89f-55fbc016bbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:05.248733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 823/1683] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jul 12 13:42:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:05.334537 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff7670-f025-4f1b-82d8-039723b180b6 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:05.359784 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff7670-f025-4f1b-82d8-039723b180b6 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:05.370174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45e178-c32a-4a07-b54d-2b72cdf8c587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:05.370864 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45e178-c32a-4a07-b54d-2b72cdf8c587 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:05.371146 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45e178-c32a-4a07-b54d-2b72cdf8c587 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:05.371421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45e178-c32a-4a07-b54d-2b72cdf8c587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:05.405791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45e178-c32a-4a07-b54d-2b72cdf8c587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:05.407759 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45e178-c32a-4a07-b54d-2b72cdf8c587 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:05.425919 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 861/1684] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:05.697430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a64d5-9311-4483-b89f-55fbc016bbb8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:05.699725 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a70a64d5-9311-4483-b89f-55fbc016bbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:05.729475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a64d5-9311-4483-b89f-55fbc016bbb8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:05.740078 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496fa4d-d10a-4786-8225-360f1398f1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:05.741389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496fa4d-d10a-4786-8225-360f1398f1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:05.741675 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496fa4d-d10a-4786-8225-360f1398f1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:05.741998 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496fa4d-d10a-4786-8225-360f1398f1c9 admin admin] PATH_INFO: `/v3/users/12826723aeef4f2785a12beb27dbcb31` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:05.746091 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496fa4d-d10a-4786-8225-360f1398f1c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12826723aeef4f2785a12beb27dbcb31)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:05.748928 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496fa4d-d10a-4786-8225-360f1398f1c9 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:05.751543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:05.861291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2496fa4d-d10a-4786-8225-360f1398f1c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12826723aeef4f2785a12beb27dbcb31'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:05.895718 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2496fa4d-d10a-4786-8225-360f1398f1c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12826723aeef4f2785a12beb27dbcb31'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:05.935371 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2496fa4d-d10a-4786-8225-360f1398f1c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12826723aeef4f2785a12beb27dbcb31'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:05.937827 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2496fa4d-d10a-4786-8225-360f1398f1c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12826723aeef4f2785a12beb27dbcb31'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:05.966788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2496fa4d-d10a-4786-8225-360f1398f1c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12826723aeef4f2785a12beb27dbcb31'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:05.974486 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 824/1685] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:42:05 2022] DELETE /identity/v3/users/12826723aeef4f2785a12beb27dbcb31 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:06.150488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45e178-c32a-4a07-b54d-2b72cdf8c587 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:06.153118 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4e45e178-c32a-4a07-b54d-2b72cdf8c587 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:06.180399 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163e4f1-c8d7-4acf-a3bc-c4c9de41bee4 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:06.180692 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163e4f1-c8d7-4acf-a3bc-c4c9de41bee4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:06.181748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163e4f1-c8d7-4acf-a3bc-c4c9de41bee4 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:06.181852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45e178-c32a-4a07-b54d-2b72cdf8c587 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:06.182707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 825/1686] 10.222.0.52 () {54 vars in 912 bytes} [Tue Jul 12 13:42:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:42:06.189457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17892b3-639d-4b40-9a51-bc86ebacd652 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:06.189648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17892b3-639d-4b40-9a51-bc86ebacd652 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:06.189909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17892b3-639d-4b40-9a51-bc86ebacd652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:06.243923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:06.247274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a17892b3-639d-4b40-9a51-bc86ebacd652 None None] MFA Rules not processed for user `3e833722fe4e493e9ddc4b275632536f`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:06.248463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a17892b3-639d-4b40-9a51-bc86ebacd652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:06.368080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:06.369026 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:06.369380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:06.369732 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151 admin admin] PATH_INFO: `/v3/projects/e78f1d11bdee4728a8ed0b39b57c5eab` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:06.375173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e78f1d11bdee4728a8ed0b39b57c5eab)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:06.382315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:06.400800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 826/1687] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jul 12 13:42:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:42:06.531837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e78f1d11bdee4728a8ed0b39b57c5eab', 'request_id': 'req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:06.548136 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17892b3-639d-4b40-9a51-bc86ebacd652 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:06.550681 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a17892b3-639d-4b40-9a51-bc86ebacd652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:06.554181 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e78f1d11bdee4728a8ed0b39b57c5eab', 'request_id': 'req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:06.575291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e78f1d11bdee4728a8ed0b39b57c5eab', 'request_id': 'req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:06.604825 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151 admin admin] The token cache is being invalidate because project e78f1d11bdee4728a8ed0b39b57c5eab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:42:06.606309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:06.607601 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:06.613135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 862/1688] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:42:06 2022] DELETE /identity/v3/projects/e78f1d11bdee4728a8ed0b39b57c5eab => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:06.617805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:06.620468 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:06.754142 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17892b3-639d-4b40-9a51-bc86ebacd652 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:06.818406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb3a84-e1ac-4ae5-a373-7df7b2f4a151 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:06.824776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5943be0-b4c7-483a-9a60-e6cdf2cff64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:06.825689 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5943be0-b4c7-483a-9a60-e6cdf2cff64c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:06.825989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5943be0-b4c7-483a-9a60-e6cdf2cff64c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:06.826307 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5943be0-b4c7-483a-9a60-e6cdf2cff64c admin admin] PATH_INFO: `/v3/users/8f62a70154ee47a7964c2bbb6cc65ee8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:06.827991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5943be0-b4c7-483a-9a60-e6cdf2cff64c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f62a70154ee47a7964c2bbb6cc65ee8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:06.842308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:06.843336 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5943be0-b4c7-483a-9a60-e6cdf2cff64c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:06.904600 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73c12b-c65a-4d76-8752-29e570998766 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1308deea1ea4629a1ac22d4b903ad4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:06.905368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73c12b-c65a-4d76-8752-29e570998766 service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:06.906147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73c12b-c65a-4d76-8752-29e570998766 service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:06.906646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73c12b-c65a-4d76-8752-29e570998766 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:06.909129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6d73c12b-c65a-4d76-8752-29e570998766 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:06.927451 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73c12b-c65a-4d76-8752-29e570998766 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:06.929483 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73c12b-c65a-4d76-8752-29e570998766 service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:06.942851 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 827/1689] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:06.944129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73c12b-c65a-4d76-8752-29e570998766 service placement] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:06.946102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6d73c12b-c65a-4d76-8752-29e570998766 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:07.008485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b5943be0-b4c7-483a-9a60-e6cdf2cff64c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f62a70154ee47a7964c2bbb6cc65ee8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:07.010777 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b5943be0-b4c7-483a-9a60-e6cdf2cff64c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f62a70154ee47a7964c2bbb6cc65ee8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:07.011417 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b5943be0-b4c7-483a-9a60-e6cdf2cff64c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f62a70154ee47a7964c2bbb6cc65ee8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:07.043899 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:07.049784 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b5943be0-b4c7-483a-9a60-e6cdf2cff64c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f62a70154ee47a7964c2bbb6cc65ee8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:07.070715 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b5943be0-b4c7-483a-9a60-e6cdf2cff64c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f62a70154ee47a7964c2bbb6cc65ee8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:07.078722 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73c12b-c65a-4d76-8752-29e570998766 service placement] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:07.088518 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f4a2d-7e5c-44bb-9ddf-64390f3dc91e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:07.089293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f4a2d-7e5c-44bb-9ddf-64390f3dc91e service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:07.090536 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f4a2d-7e5c-44bb-9ddf-64390f3dc91e service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:07.090863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f4a2d-7e5c-44bb-9ddf-64390f3dc91e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:07.095636 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c27f4a2d-7e5c-44bb-9ddf-64390f3dc91e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:07.101583 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 863/1690] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:42:06 2022] DELETE /identity/v3/users/8f62a70154ee47a7964c2bbb6cc65ee8 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:07.102697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5943be0-b4c7-483a-9a60-e6cdf2cff64c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:07.104345 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b5943be0-b4c7-483a-9a60-e6cdf2cff64c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:07.141687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f4a2d-7e5c-44bb-9ddf-64390f3dc91e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:07.147025 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f4a2d-7e5c-44bb-9ddf-64390f3dc91e service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:07.180484 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 828/1691] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:07.357278 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5943be0-b4c7-483a-9a60-e6cdf2cff64c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:07.381921 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:07.382962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:07.383496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:07.387547 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60 admin admin] PATH_INFO: `/v3/projects/404b12bef8ea4d788b1e352576911b8c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:07.392340 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=404b12bef8ea4d788b1e352576911b8c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:07.397371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:07.595229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '404b12bef8ea4d788b1e352576911b8c', 'request_id': 'req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:07.617284 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '404b12bef8ea4d788b1e352576911b8c', 'request_id': 'req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:07.629391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '404b12bef8ea4d788b1e352576911b8c', 'request_id': 'req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:07.658244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60 admin admin] The token cache is being invalidate because project 404b12bef8ea4d788b1e352576911b8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:42:07.658643 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:07.659608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:07.663346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 864/1692] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:42:07 2022] DELETE /identity/v3/projects/404b12bef8ea4d788b1e352576911b8c => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:07.667391 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f4a2d-7e5c-44bb-9ddf-64390f3dc91e service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:07.687226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f4a2d-7e5c-44bb-9ddf-64390f3dc91e service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:07.895376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dce261-64f5-4c7f-8e9a-8d9aa7816a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:07.896445 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dce261-64f5-4c7f-8e9a-8d9aa7816a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:07.896813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dce261-64f5-4c7f-8e9a-8d9aa7816a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:07.897221 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dce261-64f5-4c7f-8e9a-8d9aa7816a3d admin admin] PATH_INFO: `/v3/users/a6b0d2cb54424479a6229441834dbfe8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:07.910497 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dce261-64f5-4c7f-8e9a-8d9aa7816a3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6b0d2cb54424479a6229441834dbfe8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:07.913545 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:07.914916 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dce261-64f5-4c7f-8e9a-8d9aa7816a3d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:07.915234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:07.942296 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4cb97-1b6c-45bb-a18b-911d688bbe60 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:07.959302 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5167977-4cee-4c6d-b01c-0712a3be7fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:07.960957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5167977-4cee-4c6d-b01c-0712a3be7fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:07.961242 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5167977-4cee-4c6d-b01c-0712a3be7fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:07.961523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5167977-4cee-4c6d-b01c-0712a3be7fce admin admin] PATH_INFO: `/v3/projects/f73a4312030644ea86622ffd31cb2355` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:07.964732 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5167977-4cee-4c6d-b01c-0712a3be7fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=f73a4312030644ea86622ffd31cb2355)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:07.969860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5167977-4cee-4c6d-b01c-0712a3be7fce admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:08.019937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-52dce261-64f5-4c7f-8e9a-8d9aa7816a3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6b0d2cb54424479a6229441834dbfe8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:08.036710 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-52dce261-64f5-4c7f-8e9a-8d9aa7816a3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6b0d2cb54424479a6229441834dbfe8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:08.039289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:08.050243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-52dce261-64f5-4c7f-8e9a-8d9aa7816a3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6b0d2cb54424479a6229441834dbfe8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:08.052239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-52dce261-64f5-4c7f-8e9a-8d9aa7816a3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6b0d2cb54424479a6229441834dbfe8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:08.065726 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-52dce261-64f5-4c7f-8e9a-8d9aa7816a3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6b0d2cb54424479a6229441834dbfe8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:08.099309 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 829/1693] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:42:07 2022] DELETE /identity/v3/users/a6b0d2cb54424479a6229441834dbfe8 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:08.107849 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f5167977-4cee-4c6d-b01c-0712a3be7fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f73a4312030644ea86622ffd31cb2355', 'request_id': 'req-f5167977-4cee-4c6d-b01c-0712a3be7fce'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:08.126777 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f5167977-4cee-4c6d-b01c-0712a3be7fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f73a4312030644ea86622ffd31cb2355', 'request_id': 'req-f5167977-4cee-4c6d-b01c-0712a3be7fce'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:08.140972 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f5167977-4cee-4c6d-b01c-0712a3be7fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f73a4312030644ea86622ffd31cb2355', 'request_id': 'req-f5167977-4cee-4c6d-b01c-0712a3be7fce'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:08.154604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:08.180371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f5167977-4cee-4c6d-b01c-0712a3be7fce admin admin] The token cache is being invalidate because project f73a4312030644ea86622ffd31cb2355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:42:08.183453 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f5167977-4cee-4c6d-b01c-0712a3be7fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:08.185504 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f5167977-4cee-4c6d-b01c-0712a3be7fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:08.189754 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 865/1694] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:42:07 2022] DELETE /identity/v3/projects/f73a4312030644ea86622ffd31cb2355 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:42:08.196170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90570a5a-2140-47a1-babd-9a96806f203f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:08.196669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90570a5a-2140-47a1-babd-9a96806f203f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:08.197100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90570a5a-2140-47a1-babd-9a96806f203f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:08.245245 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-90570a5a-2140-47a1-babd-9a96806f203f None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:08.246585 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-90570a5a-2140-47a1-babd-9a96806f203f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:08.439301 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 830/1695] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:42:08.449622 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80895f8-95df-48fa-9637-34a15d10da43 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:08.450751 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80895f8-95df-48fa-9637-34a15d10da43 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:08.451651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80895f8-95df-48fa-9637-34a15d10da43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:08.488441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b80895f8-95df-48fa-9637-34a15d10da43 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:08.493766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b80895f8-95df-48fa-9637-34a15d10da43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:08.533134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 866/1696] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:42:08.539040 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90570a5a-2140-47a1-babd-9a96806f203f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:08.560425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90570a5a-2140-47a1-babd-9a96806f203f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:08.561849 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80895f8-95df-48fa-9637-34a15d10da43 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:08.563621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b80895f8-95df-48fa-9637-34a15d10da43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:08.589228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc75e7c-20dc-4436-8df6-798d44447163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:08.590029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc75e7c-20dc-4436-8df6-798d44447163 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:08.590402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc75e7c-20dc-4436-8df6-798d44447163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:08.590605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc75e7c-20dc-4436-8df6-798d44447163 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:08.593105 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc75e7c-20dc-4436-8df6-798d44447163 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:42:08.596622 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc75e7c-20dc-4436-8df6-798d44447163 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:08.602811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80895f8-95df-48fa-9637-34a15d10da43 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:08.604039 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc75e7c-20dc-4436-8df6-798d44447163 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:08.624337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136b8f3-d7aa-4fe1-9d6d-0115c39a1e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:08.625309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136b8f3-d7aa-4fe1-9d6d-0115c39a1e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:08.625660 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136b8f3-d7aa-4fe1-9d6d-0115c39a1e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:08.626093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136b8f3-d7aa-4fe1-9d6d-0115c39a1e9e admin admin] PATH_INFO: `/v3/projects/f8a7610659d64826ad0fc963ec83e9ca` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:08.628545 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136b8f3-d7aa-4fe1-9d6d-0115c39a1e9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8a7610659d64826ad0fc963ec83e9ca)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:08.630786 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136b8f3-d7aa-4fe1-9d6d-0115c39a1e9e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:08.645900 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 831/1697] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:42:08.658643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc75e7c-20dc-4436-8df6-798d44447163 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:08.687405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc75e7c-20dc-4436-8df6-798d44447163 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:08.709666 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabcdf3-a50f-4ec3-a727-13851c402811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:08.714493 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabcdf3-a50f-4ec3-a727-13851c402811 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:08.714772 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabcdf3-a50f-4ec3-a727-13851c402811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:08.715053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabcdf3-a50f-4ec3-a727-13851c402811 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:08.715828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efabcdf3-a50f-4ec3-a727-13851c402811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:08.717919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efabcdf3-a50f-4ec3-a727-13851c402811 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:08.784395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f136b8f3-d7aa-4fe1-9d6d-0115c39a1e9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8a7610659d64826ad0fc963ec83e9ca', 'request_id': 'req-f136b8f3-d7aa-4fe1-9d6d-0115c39a1e9e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:08.806727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f136b8f3-d7aa-4fe1-9d6d-0115c39a1e9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8a7610659d64826ad0fc963ec83e9ca', 'request_id': 'req-f136b8f3-d7aa-4fe1-9d6d-0115c39a1e9e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:08.814968 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 832/1698] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:42:08 2022] POST /identity/v3/projects => generated 375 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:08.818795 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f136b8f3-d7aa-4fe1-9d6d-0115c39a1e9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8a7610659d64826ad0fc963ec83e9ca', 'request_id': 'req-f136b8f3-d7aa-4fe1-9d6d-0115c39a1e9e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:08.818990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabcdf3-a50f-4ec3-a727-13851c402811 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:08.872171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f136b8f3-d7aa-4fe1-9d6d-0115c39a1e9e admin admin] The token cache is being invalidate because project f8a7610659d64826ad0fc963ec83e9ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:42:08.872486 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f136b8f3-d7aa-4fe1-9d6d-0115c39a1e9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:08.873799 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f136b8f3-d7aa-4fe1-9d6d-0115c39a1e9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:08.878089 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 867/1699] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:42:08 2022] DELETE /identity/v3/projects/f8a7610659d64826ad0fc963ec83e9ca => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:08.882988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7255d9-3629-41c8-b316-ad16b1d44923 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:08.883462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7255d9-3629-41c8-b316-ad16b1d44923 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:08.883880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7255d9-3629-41c8-b316-ad16b1d44923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:08.925315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8f7255d9-3629-41c8-b316-ad16b1d44923 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:08.927982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8f7255d9-3629-41c8-b316-ad16b1d44923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:08.990407 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabcdf3-a50f-4ec3-a727-13851c402811 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:09.000727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c99463-34e0-4962-bde9-5d52f1e616cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:09.001786 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c99463-34e0-4962-bde9-5d52f1e616cc admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:09.002142 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c99463-34e0-4962-bde9-5d52f1e616cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:09.002422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c99463-34e0-4962-bde9-5d52f1e616cc admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:09.003228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c99463-34e0-4962-bde9-5d52f1e616cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:09.005197 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c99463-34e0-4962-bde9-5d52f1e616cc admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:09.062046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 868/1700] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:09.064647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e99b1-9647-42a9-afbf-b4e4199ac7b8 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:09.064950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e99b1-9647-42a9-afbf-b4e4199ac7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:09.065246 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e99b1-9647-42a9-afbf-b4e4199ac7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:09.069513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 833/1701] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:42:08 2022] POST /identity/v3/users => generated 288 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:09.070789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c99463-34e0-4962-bde9-5d52f1e616cc admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:09.077888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-95c99463-34e0-4962-bde9-5d52f1e616cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:09.108534 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-e51e99b1-9647-42a9-afbf-b4e4199ac7b8 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:09.109913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e51e99b1-9647-42a9-afbf-b4e4199ac7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:09.110129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c99463-34e0-4962-bde9-5d52f1e616cc admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:09.118311 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7bc61-a97b-4d9d-8135-a27192c59619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:09.118860 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7bc61-a97b-4d9d-8135-a27192c59619 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:09.119145 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7bc61-a97b-4d9d-8135-a27192c59619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:09.119422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7bc61-a97b-4d9d-8135-a27192c59619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:09.120173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7bc61-a97b-4d9d-8135-a27192c59619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:09.122009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 869/1702] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:09.127048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e99b1-9647-42a9-afbf-b4e4199ac7b8 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:09.131308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7bc61-a97b-4d9d-8135-a27192c59619 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:09.159961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 834/1703] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:09.161021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7bc61-a97b-4d9d-8135-a27192c59619 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:09.161814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e99b1-9647-42a9-afbf-b4e4199ac7b8 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:09.174439 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8c5c9-fe89-44af-a186-b5f3e03c4295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:09.178108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8c5c9-fe89-44af-a186-b5f3e03c4295 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:09.178669 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8c5c9-fe89-44af-a186-b5f3e03c4295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:09.179260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8c5c9-fe89-44af-a186-b5f3e03c4295 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:09.180666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8c5c9-fe89-44af-a186-b5f3e03c4295 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:42:09.181200 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8c5c9-fe89-44af-a186-b5f3e03c4295 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:09.182962 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7bc61-a97b-4d9d-8135-a27192c59619 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:09.184497 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8c5c9-fe89-44af-a186-b5f3e03c4295 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:09.197405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606efa14-713f-4c21-9125-0ea8892e271f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:09.198225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606efa14-713f-4c21-9125-0ea8892e271f admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:09.198518 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606efa14-713f-4c21-9125-0ea8892e271f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:09.198802 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606efa14-713f-4c21-9125-0ea8892e271f admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:09.199646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606efa14-713f-4c21-9125-0ea8892e271f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:09.201280 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606efa14-713f-4c21-9125-0ea8892e271f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:09.210781 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 870/1704] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:42:09.215097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8c5c9-fe89-44af-a186-b5f3e03c4295 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:09.240262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 835/1705] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:09.241386 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606efa14-713f-4c21-9125-0ea8892e271f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:09.253990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8c5c9-fe89-44af-a186-b5f3e03c4295 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:09.265775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606efa14-713f-4c21-9125-0ea8892e271f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:09.277948 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793ee96-e69c-42a7-8e95-f52d3d7f69f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:09.278758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793ee96-e69c-42a7-8e95-f52d3d7f69f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:09.279573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793ee96-e69c-42a7-8e95-f52d3d7f69f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:09.281872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793ee96-e69c-42a7-8e95-f52d3d7f69f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:09.283100 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793ee96-e69c-42a7-8e95-f52d3d7f69f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:09.286673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793ee96-e69c-42a7-8e95-f52d3d7f69f2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:09.308119 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b85bf4-117a-476e-b44f-f7a9f21f360f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:09.309047 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b85bf4-117a-476e-b44f-f7a9f21f360f admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:09.309333 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b85bf4-117a-476e-b44f-f7a9f21f360f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:09.326402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b85bf4-117a-476e-b44f-f7a9f21f360f admin admin] PATH_INFO: `/v3/projects/aff66bc3bdb24fd4be85e282fcb480ea/users/8039d9c7fca546ebb3ca46333cba968a/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:09.346604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 871/1706] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:42:09 2022] POST /identity/v3/projects => generated 353 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:09.348276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793ee96-e69c-42a7-8e95-f52d3d7f69f2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:09.370580 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:09.380834 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b85bf4-117a-476e-b44f-f7a9f21f360f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aff66bc3bdb24fd4be85e282fcb480ea, user_id=8039d9c7fca546ebb3ca46333cba968a, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:09.391378 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b85bf4-117a-476e-b44f-f7a9f21f360f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:09.392313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:09.432049 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 836/1707] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:42:09 2022] PUT /identity/v3/projects/aff66bc3bdb24fd4be85e282fcb480ea/users/8039d9c7fca546ebb3ca46333cba968a/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:09.442438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb16c5e-8306-4d27-aa63-c9056de12c19 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:09.442727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb16c5e-8306-4d27-aa63-c9056de12c19 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:09.443094 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb16c5e-8306-4d27-aa63-c9056de12c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:09.476659 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793ee96-e69c-42a7-8e95-f52d3d7f69f2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:09.526149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-fbb16c5e-8306-4d27-aa63-c9056de12c19 None None] MFA Rules not processed for user `8039d9c7fca546ebb3ca46333cba968a`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:09.530213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fbb16c5e-8306-4d27-aa63-c9056de12c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:09.655608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51356f9-1555-4e50-8257-975f2f4031a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:09.656571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51356f9-1555-4e50-8257-975f2f4031a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:09.656681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51356f9-1555-4e50-8257-975f2f4031a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:09.656885 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51356f9-1555-4e50-8257-975f2f4031a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:09.660996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51356f9-1555-4e50-8257-975f2f4031a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:09.666660 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51356f9-1555-4e50-8257-975f2f4031a2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:09.736577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 872/1708] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:42:09 2022] POST /identity/v3/users => generated 277 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:09.743250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51356f9-1555-4e50-8257-975f2f4031a2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:09.757304 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 837/1709] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:09.771145 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb16c5e-8306-4d27-aa63-c9056de12c19 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:09.777327 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fbb16c5e-8306-4d27-aa63-c9056de12c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:09.789891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51356f9-1555-4e50-8257-975f2f4031a2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:09.806708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d48038-7303-454a-a9d6-2597690e5808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:09.807513 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d48038-7303-454a-a9d6-2597690e5808 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:09.821579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d48038-7303-454a-a9d6-2597690e5808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:09.825170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d48038-7303-454a-a9d6-2597690e5808 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:09.826016 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d48038-7303-454a-a9d6-2597690e5808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:09.835732 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d48038-7303-454a-a9d6-2597690e5808 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:09.867719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 873/1710] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:09.870309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0e226-00d2-4fd3-9240-b517d2a4ba2f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:09.870928 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0e226-00d2-4fd3-9240-b517d2a4ba2f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:09.871200 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0e226-00d2-4fd3-9240-b517d2a4ba2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:09.947836 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-e8a0e226-00d2-4fd3-9240-b517d2a4ba2f None None] MFA Rules not processed for user `2ad8ebe54a1d42eda9b23108f92a1d23`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:09.950266 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e8a0e226-00d2-4fd3-9240-b517d2a4ba2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:10.068450 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb16c5e-8306-4d27-aa63-c9056de12c19 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:10.075035 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f302cea-2de0-491a-8441-100745aff6e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebb4da7b00c64fafbb3724bae25b2853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:10.076106 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f302cea-2de0-491a-8441-100745aff6e1 service glance] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:10.076542 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f302cea-2de0-491a-8441-100745aff6e1 service glance] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:10.080489 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f302cea-2de0-491a-8441-100745aff6e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:10.092402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f302cea-2de0-491a-8441-100745aff6e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:10.097220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f302cea-2de0-491a-8441-100745aff6e1 service glance] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:10.113405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 838/1711] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:42:10.114809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c66817-5fd6-42d5-833b-225a761ce06a None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:10.115099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c66817-5fd6-42d5-833b-225a761ce06a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:10.115378 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c66817-5fd6-42d5-833b-225a761ce06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:10.141087 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b2c66817-5fd6-42d5-833b-225a761ce06a None None] MFA Rules not processed for user `4a6b3bd40ad744908a05df7410cace52`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:10.142365 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b2c66817-5fd6-42d5-833b-225a761ce06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:10.230325 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 874/1712] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:10.245113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0e226-00d2-4fd3-9240-b517d2a4ba2f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:10.299262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0e226-00d2-4fd3-9240-b517d2a4ba2f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:10.322215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cdf47-3f91-48ab-8d28-d77aa83ec9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:10.322215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cdf47-3f91-48ab-8d28-d77aa83ec9ef admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:10.322215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cdf47-3f91-48ab-8d28-d77aa83ec9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:10.322215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cdf47-3f91-48ab-8d28-d77aa83ec9ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:10.322619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5cdf47-3f91-48ab-8d28-d77aa83ec9ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:10.339175 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5cdf47-3f91-48ab-8d28-d77aa83ec9ef admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:10.348131 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 839/1713] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:10.351281 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c66817-5fd6-42d5-833b-225a761ce06a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:10.372796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b2c66817-5fd6-42d5-833b-225a761ce06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:10.395908 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 875/1714] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:10.398516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cdf47-3f91-48ab-8d28-d77aa83ec9ef admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:10.420841 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:10.432963 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cdf47-3f91-48ab-8d28-d77aa83ec9ef admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:10.456301 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf026b-80a5-43e3-9c6c-f8e6794d7f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:10.457438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf026b-80a5-43e3-9c6c-f8e6794d7f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:10.461364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf026b-80a5-43e3-9c6c-f8e6794d7f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:10.461633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf026b-80a5-43e3-9c6c-f8e6794d7f6f admin admin] PATH_INFO: `/v3/projects/bfb29d7beffd4bd89dae6bc42c0b7296/users/aad547342496410e8445da52cfb43f5b/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:10.572177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf026b-80a5-43e3-9c6c-f8e6794d7f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfb29d7beffd4bd89dae6bc42c0b7296, user_id=aad547342496410e8445da52cfb43f5b, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:10.597772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf026b-80a5-43e3-9c6c-f8e6794d7f6f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:10.700286 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 876/1715] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:42:10 2022] PUT /identity/v3/projects/bfb29d7beffd4bd89dae6bc42c0b7296/users/aad547342496410e8445da52cfb43f5b/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:10.700286 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a100b-01ca-4edd-a234-a731889f3a0d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:10.700286 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a100b-01ca-4edd-a234-a731889f3a0d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:10.700286 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a100b-01ca-4edd-a234-a731889f3a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:10.728396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c66817-5fd6-42d5-833b-225a761ce06a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:10.744173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:10.943723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-424a100b-01ca-4edd-a234-a731889f3a0d None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:10.949720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-424a100b-01ca-4edd-a234-a731889f3a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:11.044766 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2b56b-e4f2-4dab-8a85-917e2af33693 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:11.045448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2b56b-e4f2-4dab-8a85-917e2af33693 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:11.045733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2b56b-e4f2-4dab-8a85-917e2af33693 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:11.055877 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2b56b-e4f2-4dab-8a85-917e2af33693 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:11.253241 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 877/1716] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jul 12 13:42:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 561 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:11.257355 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ec8b0-6504-4bb8-90a7-beb76520a10a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:11.258008 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ec8b0-6504-4bb8-90a7-beb76520a10a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:11.259801 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ec8b0-6504-4bb8-90a7-beb76520a10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:11.283349 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2b56b-e4f2-4dab-8a85-917e2af33693 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:11.293610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2b56b-e4f2-4dab-8a85-917e2af33693 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:11.337621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 840/1717] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 989 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:11.401749 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-870ec8b0-6504-4bb8-90a7-beb76520a10a None None] MFA Rules not processed for user `aad547342496410e8445da52cfb43f5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:11.415113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-870ec8b0-6504-4bb8-90a7-beb76520a10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:11.664104 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 878/1718] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:11 2022] POST /identity/v3/auth/tokens => generated 3220 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:11.666915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2b56b-e4f2-4dab-8a85-917e2af33693 service nova] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:11.673047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ec8b0-6504-4bb8-90a7-beb76520a10a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:11.676782 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-870ec8b0-6504-4bb8-90a7-beb76520a10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:11.938158 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:11.977103 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:12.036628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2b56b-e4f2-4dab-8a85-917e2af33693 service nova] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:12.055743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ec8b0-6504-4bb8-90a7-beb76520a10a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:12.062666 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87b332-5afe-4c67-90a9-11a09938ea09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:12.066681 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87b332-5afe-4c67-90a9-11a09938ea09 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:12.067025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4615b94-1487-4e8f-9a5f-d33d4f330caf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:12.067427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87b332-5afe-4c67-90a9-11a09938ea09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:12.069671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87b332-5afe-4c67-90a9-11a09938ea09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:12.070111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4615b94-1487-4e8f-9a5f-d33d4f330caf service swift] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:12.072598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4615b94-1487-4e8f-9a5f-d33d4f330caf service swift] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:12.073011 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4615b94-1487-4e8f-9a5f-d33d4f330caf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:12.073707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87b332-5afe-4c67-90a9-11a09938ea09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:12.081447 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87b332-5afe-4c67-90a9-11a09938ea09 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:12.103233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4615b94-1487-4e8f-9a5f-d33d4f330caf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:12.121110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4615b94-1487-4e8f-9a5f-d33d4f330caf service swift] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:12.165617 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 879/1719] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:42:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 13:42:12.230470 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 841/1720] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:42:11 2022] POST /identity/v3/projects => generated 357 bytes in 565 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:42:12.234061 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4615b94-1487-4e8f-9a5f-d33d4f330caf service swift] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:12.277779 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4615b94-1487-4e8f-9a5f-d33d4f330caf service swift] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:12.694260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3204a5d-391f-4fb5-ad77-8a772b535ae1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:12.694771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3204a5d-391f-4fb5-ad77-8a772b535ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:12.695798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3204a5d-391f-4fb5-ad77-8a772b535ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:12.722426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc60d5-c40f-4098-b837-062f1f02b29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:12.728785 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc60d5-c40f-4098-b837-062f1f02b29a admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:12.731727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc60d5-c40f-4098-b837-062f1f02b29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:12.734392 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc60d5-c40f-4098-b837-062f1f02b29a admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:12.742233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc60d5-c40f-4098-b837-062f1f02b29a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:12.753760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc60d5-c40f-4098-b837-062f1f02b29a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:12.790407 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d3204a5d-391f-4fb5-ad77-8a772b535ae1 None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:12.806704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d3204a5d-391f-4fb5-ad77-8a772b535ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:12.868057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 880/1721] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:42:12 2022] POST /identity/v3/users => generated 285 bytes in 634 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:12.873274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc60d5-c40f-4098-b837-062f1f02b29a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:12.922812 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc60d5-c40f-4098-b837-062f1f02b29a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:12.938166 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716434d2-df48-4653-87ca-59fc1303021c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:12.943909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716434d2-df48-4653-87ca-59fc1303021c admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:12.944393 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716434d2-df48-4653-87ca-59fc1303021c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:12.946078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716434d2-df48-4653-87ca-59fc1303021c admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:12.950343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716434d2-df48-4653-87ca-59fc1303021c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:12.959037 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716434d2-df48-4653-87ca-59fc1303021c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:12.994735 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 881/1722] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:13.002644 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716434d2-df48-4653-87ca-59fc1303021c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:13.045622 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716434d2-df48-4653-87ca-59fc1303021c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:13.057807 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 842/1723] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jul 12 13:42:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:13.069908 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2218b6c-95b2-4a31-b03b-251e1a980ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:13.071273 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2218b6c-95b2-4a31-b03b-251e1a980ded admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:13.076314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2218b6c-95b2-4a31-b03b-251e1a980ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:13.077374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2218b6c-95b2-4a31-b03b-251e1a980ded admin admin] PATH_INFO: `/v3/projects/514ef28d8dd441c8a3812d8e040f7618/users/7c5f2d61711b4b2ab9b70b16eef683b3/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:13.086022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3204a5d-391f-4fb5-ad77-8a772b535ae1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:13.121317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2218b6c-95b2-4a31-b03b-251e1a980ded admin admin] RBAC: Authorizing `identity:create_grant(project_id=514ef28d8dd441c8a3812d8e040f7618, user_id=7c5f2d61711b4b2ab9b70b16eef683b3, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:13.127262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2218b6c-95b2-4a31-b03b-251e1a980ded admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:13.180841 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 882/1724] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:42:13 2022] PUT /identity/v3/projects/514ef28d8dd441c8a3812d8e040f7618/users/7c5f2d61711b4b2ab9b70b16eef683b3/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:13.183425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2218b6c-95b2-4a31-b03b-251e1a980ded admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:13.290494 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3204a5d-391f-4fb5-ad77-8a772b535ae1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:13.363695 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2218b6c-95b2-4a31-b03b-251e1a980ded admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:13.391173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de98ff-ae08-4c2a-b795-198d410c62dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:13.393416 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de98ff-ae08-4c2a-b795-198d410c62dd admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:13.393714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de98ff-ae08-4c2a-b795-198d410c62dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:13.395479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de98ff-ae08-4c2a-b795-198d410c62dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:13.396279 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de98ff-ae08-4c2a-b795-198d410c62dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:13.402296 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de98ff-ae08-4c2a-b795-198d410c62dd admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:13.437512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 883/1725] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:13.438740 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de98ff-ae08-4c2a-b795-198d410c62dd admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:13.486427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de98ff-ae08-4c2a-b795-198d410c62dd admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:13.487663 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c53b5a-26d2-457e-8268-3f74d45088ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:13.494875 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c53b5a-26d2-457e-8268-3f74d45088ef service swift] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:13.495173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c53b5a-26d2-457e-8268-3f74d45088ef service swift] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:13.495843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c53b5a-26d2-457e-8268-3f74d45088ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:13.506212 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062bebaf-de2c-4963-8382-63921a62e355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:13.508040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062bebaf-de2c-4963-8382-63921a62e355 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:13.508329 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062bebaf-de2c-4963-8382-63921a62e355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:13.508642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062bebaf-de2c-4963-8382-63921a62e355 admin admin] PATH_INFO: `/v3/projects/514ef28d8dd441c8a3812d8e040f7618/users/7c5f2d61711b4b2ab9b70b16eef683b3/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:13.517913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062bebaf-de2c-4963-8382-63921a62e355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=514ef28d8dd441c8a3812d8e040f7618, user_id=7c5f2d61711b4b2ab9b70b16eef683b3, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:13.520695 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062bebaf-de2c-4963-8382-63921a62e355 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:13.570515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:13.594599 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 884/1726] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:42:13 2022] PUT /identity/v3/projects/514ef28d8dd441c8a3812d8e040f7618/users/7c5f2d61711b4b2ab9b70b16eef683b3/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:13.604899 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087139b5-9790-4b0b-91ab-4cf487e50e4f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:13.605261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087139b5-9790-4b0b-91ab-4cf487e50e4f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:13.605528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087139b5-9790-4b0b-91ab-4cf487e50e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:13.734048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-087139b5-9790-4b0b-91ab-4cf487e50e4f None None] MFA Rules not processed for user `7c5f2d61711b4b2ab9b70b16eef683b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:13.739295 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-087139b5-9790-4b0b-91ab-4cf487e50e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:13.763147 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:13.767194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c53b5a-26d2-457e-8268-3f74d45088ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:13.783950 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c53b5a-26d2-457e-8268-3f74d45088ef service swift] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:13.829391 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 843/1727] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:42:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 742 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 13:42:14.090085 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 885/1728] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:13 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:14.103635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c53b5a-26d2-457e-8268-3f74d45088ef service swift] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:14.105072 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-99c53b5a-26d2-457e-8268-3f74d45088ef service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:14.273865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c53b5a-26d2-457e-8268-3f74d45088ef service swift] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:14.408821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59aa848-f03b-4e1c-884a-e26e16706a79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:14.409885 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59aa848-f03b-4e1c-884a-e26e16706a79 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:14.410411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59aa848-f03b-4e1c-884a-e26e16706a79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:14.411059 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59aa848-f03b-4e1c-884a-e26e16706a79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:14.439680 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59aa848-f03b-4e1c-884a-e26e16706a79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:14.441337 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59aa848-f03b-4e1c-884a-e26e16706a79 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:14.468217 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 844/1729] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:14 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:14.674451 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087139b5-9790-4b0b-91ab-4cf487e50e4f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:14.690474 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087139b5-9790-4b0b-91ab-4cf487e50e4f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:14.705136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65105e15-49b9-4bf5-b3cb-c92da128fdc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:14.705815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65105e15-49b9-4bf5-b3cb-c92da128fdc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:14.708604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65105e15-49b9-4bf5-b3cb-c92da128fdc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:14.710499 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65105e15-49b9-4bf5-b3cb-c92da128fdc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:14.908319 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65105e15-49b9-4bf5-b3cb-c92da128fdc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:14.910729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65105e15-49b9-4bf5-b3cb-c92da128fdc6 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:14.950854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 886/1730] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:14 2022] GET /identity/v3/auth/tokens => generated 3220 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:15.978054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291711c1-3ce6-42a8-bc17-07e37a71dc2d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:15.978645 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291711c1-3ce6-42a8-bc17-07e37a71dc2d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:15.979100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291711c1-3ce6-42a8-bc17-07e37a71dc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:16.039741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-291711c1-3ce6-42a8-bc17-07e37a71dc2d None None] MFA Rules not processed for user `3e833722fe4e493e9ddc4b275632536f`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:16.050121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-291711c1-3ce6-42a8-bc17-07e37a71dc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:16.143399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf57c0d-d969-45c1-91fc-0ac0b5b836ed None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:16.143983 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf57c0d-d969-45c1-91fc-0ac0b5b836ed None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:16.146627 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf57c0d-d969-45c1-91fc-0ac0b5b836ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:16.223861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-cdf57c0d-d969-45c1-91fc-0ac0b5b836ed None None] MFA Rules not processed for user `a892b2511463404c818c63e870a23389`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:16.232655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cdf57c0d-d969-45c1-91fc-0ac0b5b836ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:16.323015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 845/1731] 10.222.0.50 () {58 vars in 1015 bytes} [Tue Jul 12 13:42:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:16.447822 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 887/1732] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:16.504814 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291711c1-3ce6-42a8-bc17-07e37a71dc2d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:16.640717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:16.749635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291711c1-3ce6-42a8-bc17-07e37a71dc2d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:16.764330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c2861-68f2-480f-9956-2e8acc177c14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:16.765418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c2861-68f2-480f-9956-2e8acc177c14 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:16.765724 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c2861-68f2-480f-9956-2e8acc177c14 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:16.770537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c2861-68f2-480f-9956-2e8acc177c14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:16.794980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c2861-68f2-480f-9956-2e8acc177c14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:16.797474 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c2861-68f2-480f-9956-2e8acc177c14 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:16.858617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 846/1733] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:17.052821 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf57c0d-d969-45c1-91fc-0ac0b5b836ed None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:17.054492 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cdf57c0d-d969-45c1-91fc-0ac0b5b836ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:17.071348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf57c0d-d969-45c1-91fc-0ac0b5b836ed None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:17.085702 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd58b01-6775-4abf-9bf6-c65eb5007875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:17.086849 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd58b01-6775-4abf-9bf6-c65eb5007875 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:17.087442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd58b01-6775-4abf-9bf6-c65eb5007875 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:17.088177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd58b01-6775-4abf-9bf6-c65eb5007875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:17.098169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd58b01-6775-4abf-9bf6-c65eb5007875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:17.100617 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd58b01-6775-4abf-9bf6-c65eb5007875 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:17.113184 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 888/1734] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:17.345908 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e205c-eacc-4a88-b769-e43895aea973 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:17.346361 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e205c-eacc-4a88-b769-e43895aea973 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:17.346537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e205c-eacc-4a88-b769-e43895aea973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:17.424249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-fb7e205c-eacc-4a88-b769-e43895aea973 None None] MFA Rules not processed for user `d9e5bc258de54fd6b3811401783a7408`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:17.428357 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fb7e205c-eacc-4a88-b769-e43895aea973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:17.753891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 847/1735] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:42:17.794335 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd58b01-6775-4abf-9bf6-c65eb5007875 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:17.822926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd58b01-6775-4abf-9bf6-c65eb5007875 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:17.868581 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984cfce-6294-4b97-a896-bbe0050f6956 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:17.869659 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984cfce-6294-4b97-a896-bbe0050f6956 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:17.870184 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984cfce-6294-4b97-a896-bbe0050f6956 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:17.870679 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984cfce-6294-4b97-a896-bbe0050f6956 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:17.886759 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c984cfce-6294-4b97-a896-bbe0050f6956 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:17.891666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c984cfce-6294-4b97-a896-bbe0050f6956 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:17.918040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 889/1736] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:21.965321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f2cc2-9358-4b79-964f-5fc857fac746 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:21.965875 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f2cc2-9358-4b79-964f-5fc857fac746 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:21.965875 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f2cc2-9358-4b79-964f-5fc857fac746 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:21.967560 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 848/1737] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jul 12 13:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:42:22.035666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954f01f-1016-489e-a8ff-fd8ccf59d227 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:22.036207 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954f01f-1016-489e-a8ff-fd8ccf59d227 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:22.036393 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954f01f-1016-489e-a8ff-fd8ccf59d227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:22.050138 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:22.088234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1954f01f-1016-489e-a8ff-fd8ccf59d227 None None] MFA Rules not processed for user `0d3915cde0d04bcfb9b5ce8c89af1e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:22.091262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1954f01f-1016-489e-a8ff-fd8ccf59d227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:22.130399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 890/1738] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jul 12 13:42:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:22.449086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f2cc2-9358-4b79-964f-5fc857fac746 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:22.477892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f2cc2-9358-4b79-964f-5fc857fac746 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:22.497059 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52e96f-883f-4ea8-b5b9-ab42e93f5c95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:22.498963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52e96f-883f-4ea8-b5b9-ab42e93f5c95 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:22.499431 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52e96f-883f-4ea8-b5b9-ab42e93f5c95 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:22.499976 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52e96f-883f-4ea8-b5b9-ab42e93f5c95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:22.530124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52e96f-883f-4ea8-b5b9-ab42e93f5c95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:22.531960 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52e96f-883f-4ea8-b5b9-ab42e93f5c95 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:22.578906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 849/1739] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:24.329782 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3b4e2-748d-4f61-8d51-cc64e4ba85f1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:24.330329 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3b4e2-748d-4f61-8d51-cc64e4ba85f1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:24.330426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3b4e2-748d-4f61-8d51-cc64e4ba85f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:24.401850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-70d3b4e2-748d-4f61-8d51-cc64e4ba85f1 None None] MFA Rules not processed for user `2ad8ebe54a1d42eda9b23108f92a1d23`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:24.403337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-70d3b4e2-748d-4f61-8d51-cc64e4ba85f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:24.642080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 891/1740] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:26.545568 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e7e23-b106-4433-a366-76239e427c4d None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:26.546077 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e7e23-b106-4433-a366-76239e427c4d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:26.546229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e7e23-b106-4433-a366-76239e427c4d None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:26.547891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 850/1741] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jul 12 13:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:42:26.753895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7c980-1dfa-4575-842f-e7ee3c6a214b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:26.754414 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7c980-1dfa-4575-842f-e7ee3c6a214b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:26.754483 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7c980-1dfa-4575-842f-e7ee3c6a214b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:26.792654 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d0d7c980-1dfa-4575-842f-e7ee3c6a214b None None] MFA Rules not processed for user `0d3915cde0d04bcfb9b5ce8c89af1e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:26.796123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d0d7c980-1dfa-4575-842f-e7ee3c6a214b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:26.851818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 892/1742] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jul 12 13:42:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:27.845222 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e7e23-b106-4433-a366-76239e427c4d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:27.848387 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9d5e7e23-b106-4433-a366-76239e427c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:27.908746 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e7e23-b106-4433-a366-76239e427c4d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:27.945332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614cef2-68b7-45e0-9ba1-c325d479e74e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:27.951355 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614cef2-68b7-45e0-9ba1-c325d479e74e service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:27.955037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614cef2-68b7-45e0-9ba1-c325d479e74e service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:27.956362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614cef2-68b7-45e0-9ba1-c325d479e74e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:28.021037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614cef2-68b7-45e0-9ba1-c325d479e74e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:28.025500 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614cef2-68b7-45e0-9ba1-c325d479e74e service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:28.059718 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 851/1743] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:29.946978 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfadaa-a0cb-4a89-b26d-b4fe59723af9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:29.947515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfadaa-a0cb-4a89-b26d-b4fe59723af9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:29.947515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfadaa-a0cb-4a89-b26d-b4fe59723af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:29.965174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a4dfadaa-a0cb-4a89-b26d-b4fe59723af9 None None] MFA Rules not processed for user `d9e5bc258de54fd6b3811401783a7408`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:29.966518 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a4dfadaa-a0cb-4a89-b26d-b4fe59723af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:29.973708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 893/1744] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:30.195982 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35de120-e502-48fd-a4e8-2974feabd14a None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:30.196488 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35de120-e502-48fd-a4e8-2974feabd14a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:30.196581 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35de120-e502-48fd-a4e8-2974feabd14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:30.220293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f35de120-e502-48fd-a4e8-2974feabd14a None None] MFA Rules not processed for user `a892b2511463404c818c63e870a23389`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:30.221532 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f35de120-e502-48fd-a4e8-2974feabd14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:30.249493 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 852/1745] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:36.998600 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfadaa-a0cb-4a89-b26d-b4fe59723af9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:37.066807 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfadaa-a0cb-4a89-b26d-b4fe59723af9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:37.072110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3329c-da54-444b-ab60-dd9ab7277dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:37.073190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3329c-da54-444b-ab60-dd9ab7277dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:37.073335 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3329c-da54-444b-ab60-dd9ab7277dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:37.073610 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3329c-da54-444b-ab60-dd9ab7277dee admin admin] PATH_INFO: `/v3/users/aad547342496410e8445da52cfb43f5b` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:37.074877 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3329c-da54-444b-ab60-dd9ab7277dee admin admin] RBAC: Authorizing `identity:delete_user(user_id=aad547342496410e8445da52cfb43f5b)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:37.076924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3329c-da54-444b-ab60-dd9ab7277dee admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:37.128118 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-dbb3329c-da54-444b-ab60-dd9ab7277dee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aad547342496410e8445da52cfb43f5b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:37.128806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-dbb3329c-da54-444b-ab60-dd9ab7277dee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aad547342496410e8445da52cfb43f5b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:37.129421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-dbb3329c-da54-444b-ab60-dd9ab7277dee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aad547342496410e8445da52cfb43f5b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:37.143048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-dbb3329c-da54-444b-ab60-dd9ab7277dee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aad547342496410e8445da52cfb43f5b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:37.150099 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-dbb3329c-da54-444b-ab60-dd9ab7277dee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aad547342496410e8445da52cfb43f5b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:37.165809 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 894/1746] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:42:36 2022] DELETE /identity/v3/users/aad547342496410e8445da52cfb43f5b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:37.170188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed0a83-0a2f-4ae8-93c7-eef0cbaa2a8f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:37.170683 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed0a83-0a2f-4ae8-93c7-eef0cbaa2a8f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:37.170829 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed0a83-0a2f-4ae8-93c7-eef0cbaa2a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:37.190552 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-32ed0a83-0a2f-4ae8-93c7-eef0cbaa2a8f None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:37.191798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-32ed0a83-0a2f-4ae8-93c7-eef0cbaa2a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:37.253422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 853/1747] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:37.263238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3329c-da54-444b-ab60-dd9ab7277dee admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:37.264934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-dbb3329c-da54-444b-ab60-dd9ab7277dee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:37.275212 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:37.364825 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3329c-da54-444b-ab60-dd9ab7277dee admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:37.370501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2c643-91d9-44e8-ba10-174a42520cf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:37.371379 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2c643-91d9-44e8-ba10-174a42520cf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:37.371883 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2c643-91d9-44e8-ba10-174a42520cf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:37.372360 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2c643-91d9-44e8-ba10-174a42520cf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:37.386805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2c643-91d9-44e8-ba10-174a42520cf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:37.388393 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2c643-91d9-44e8-ba10-174a42520cf7 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:37.401130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 895/1748] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:38.059658 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed0a83-0a2f-4ae8-93c7-eef0cbaa2a8f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:38.061625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-32ed0a83-0a2f-4ae8-93c7-eef0cbaa2a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:38.074945 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed0a83-0a2f-4ae8-93c7-eef0cbaa2a8f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:38.080621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9d9fd-06cf-4e29-a817-cda7da050634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:38.081510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9d9fd-06cf-4e29-a817-cda7da050634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:38.081798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9d9fd-06cf-4e29-a817-cda7da050634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:38.082129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9d9fd-06cf-4e29-a817-cda7da050634 admin admin] PATH_INFO: `/v3/projects/bfb29d7beffd4bd89dae6bc42c0b7296` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:38.083406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9d9fd-06cf-4e29-a817-cda7da050634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfb29d7beffd4bd89dae6bc42c0b7296)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:38.089021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9d9fd-06cf-4e29-a817-cda7da050634 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:38.092892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:38.147000 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-91f9d9fd-06cf-4e29-a817-cda7da050634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfb29d7beffd4bd89dae6bc42c0b7296', 'request_id': 'req-91f9d9fd-06cf-4e29-a817-cda7da050634'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:38.152762 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-91f9d9fd-06cf-4e29-a817-cda7da050634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfb29d7beffd4bd89dae6bc42c0b7296', 'request_id': 'req-91f9d9fd-06cf-4e29-a817-cda7da050634'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:38.161264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-91f9d9fd-06cf-4e29-a817-cda7da050634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfb29d7beffd4bd89dae6bc42c0b7296', 'request_id': 'req-91f9d9fd-06cf-4e29-a817-cda7da050634'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:38.174767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-91f9d9fd-06cf-4e29-a817-cda7da050634 admin admin] The token cache is being invalidate because project bfb29d7beffd4bd89dae6bc42c0b7296 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:42:38.175443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-91f9d9fd-06cf-4e29-a817-cda7da050634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:38.175946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-91f9d9fd-06cf-4e29-a817-cda7da050634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:38.179063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 854/1749] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:42:38 2022] DELETE /identity/v3/projects/bfb29d7beffd4bd89dae6bc42c0b7296 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:38.182901 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2c643-91d9-44e8-ba10-174a42520cf7 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:38.194977 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2c643-91d9-44e8-ba10-174a42520cf7 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:38.313935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25b716-24e7-45e2-9acf-b540974396fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:38.315370 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25b716-24e7-45e2-9acf-b540974396fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:38.316034 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25b716-24e7-45e2-9acf-b540974396fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:38.316421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25b716-24e7-45e2-9acf-b540974396fb admin admin] PATH_INFO: `/v3/users/7c5f2d61711b4b2ab9b70b16eef683b3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:38.318002 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25b716-24e7-45e2-9acf-b540974396fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c5f2d61711b4b2ab9b70b16eef683b3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:38.320286 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25b716-24e7-45e2-9acf-b540974396fb admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:38.382790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8c25b716-24e7-45e2-9acf-b540974396fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c5f2d61711b4b2ab9b70b16eef683b3'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:38.383839 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8c25b716-24e7-45e2-9acf-b540974396fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c5f2d61711b4b2ab9b70b16eef683b3'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:38.384319 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8c25b716-24e7-45e2-9acf-b540974396fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c5f2d61711b4b2ab9b70b16eef683b3'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:38.414491 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8c25b716-24e7-45e2-9acf-b540974396fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c5f2d61711b4b2ab9b70b16eef683b3'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:38.423694 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8c25b716-24e7-45e2-9acf-b540974396fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c5f2d61711b4b2ab9b70b16eef683b3'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:38.447902 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 896/1750] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:42:38 2022] DELETE /identity/v3/users/7c5f2d61711b4b2ab9b70b16eef683b3 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:42:39.355817 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9d9fd-06cf-4e29-a817-cda7da050634 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:39.367766 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-91f9d9fd-06cf-4e29-a817-cda7da050634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:39.440145 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9d9fd-06cf-4e29-a817-cda7da050634 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:39.536540 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afae15-2c8b-44fe-8463-ca3cb6f19fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:39.538080 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afae15-2c8b-44fe-8463-ca3cb6f19fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:39.538463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afae15-2c8b-44fe-8463-ca3cb6f19fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:39.538984 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afae15-2c8b-44fe-8463-ca3cb6f19fee admin admin] PATH_INFO: `/v3/projects/514ef28d8dd441c8a3812d8e040f7618` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:39.540439 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afae15-2c8b-44fe-8463-ca3cb6f19fee admin admin] RBAC: Authorizing `identity:delete_project(project_id=514ef28d8dd441c8a3812d8e040f7618)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:39.542537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afae15-2c8b-44fe-8463-ca3cb6f19fee admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:39.633736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d4afae15-2c8b-44fe-8463-ca3cb6f19fee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '514ef28d8dd441c8a3812d8e040f7618', 'request_id': 'req-d4afae15-2c8b-44fe-8463-ca3cb6f19fee'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:39.641226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d4afae15-2c8b-44fe-8463-ca3cb6f19fee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '514ef28d8dd441c8a3812d8e040f7618', 'request_id': 'req-d4afae15-2c8b-44fe-8463-ca3cb6f19fee'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:39.659981 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d4afae15-2c8b-44fe-8463-ca3cb6f19fee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '514ef28d8dd441c8a3812d8e040f7618', 'request_id': 'req-d4afae15-2c8b-44fe-8463-ca3cb6f19fee'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:39.678668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d4afae15-2c8b-44fe-8463-ca3cb6f19fee admin admin] The token cache is being invalidate because project 514ef28d8dd441c8a3812d8e040f7618 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:42:39.678959 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d4afae15-2c8b-44fe-8463-ca3cb6f19fee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:39.680284 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d4afae15-2c8b-44fe-8463-ca3cb6f19fee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:39.685047 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 855/1751] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:42:39 2022] DELETE /identity/v3/projects/514ef28d8dd441c8a3812d8e040f7618 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:39.687047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146907ef-ae9b-4bfe-bef4-eb58404ee19f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:39.687615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146907ef-ae9b-4bfe-bef4-eb58404ee19f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:39.688095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146907ef-ae9b-4bfe-bef4-eb58404ee19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:39.736060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-146907ef-ae9b-4bfe-bef4-eb58404ee19f None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:39.737709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-146907ef-ae9b-4bfe-bef4-eb58404ee19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:39.767696 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:39.811451 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 897/1752] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:39.819949 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd55c2d-5b1c-426c-a6e7-7db04db1051b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:39.820239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd55c2d-5b1c-426c-a6e7-7db04db1051b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:39.820574 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd55c2d-5b1c-426c-a6e7-7db04db1051b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:39.853691 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-bbd55c2d-5b1c-426c-a6e7-7db04db1051b None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:39.855233 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bbd55c2d-5b1c-426c-a6e7-7db04db1051b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:39.866457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 856/1753] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:39.871109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146907ef-ae9b-4bfe-bef4-eb58404ee19f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:39.886247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146907ef-ae9b-4bfe-bef4-eb58404ee19f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:39.904558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1809e7-c5ef-4ae3-9ef2-ed0ac121cd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:39.905250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1809e7-c5ef-4ae3-9ef2-ed0ac121cd44 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:39.905538 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1809e7-c5ef-4ae3-9ef2-ed0ac121cd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:39.905818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1809e7-c5ef-4ae3-9ef2-ed0ac121cd44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:39.906584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1809e7-c5ef-4ae3-9ef2-ed0ac121cd44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:42:39.906898 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1809e7-c5ef-4ae3-9ef2-ed0ac121cd44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:39.908525 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1809e7-c5ef-4ae3-9ef2-ed0ac121cd44 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:39.933689 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 898/1754] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:42:39.940875 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd55c2d-5b1c-426c-a6e7-7db04db1051b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:39.950811 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd55c2d-5b1c-426c-a6e7-7db04db1051b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:39.955723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01635075-8e95-4c77-96f8-6ff9ca21b502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:39.956387 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01635075-8e95-4c77-96f8-6ff9ca21b502 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:39.956664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01635075-8e95-4c77-96f8-6ff9ca21b502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:39.956936 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01635075-8e95-4c77-96f8-6ff9ca21b502 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:39.957706 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01635075-8e95-4c77-96f8-6ff9ca21b502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:39.960412 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01635075-8e95-4c77-96f8-6ff9ca21b502 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:39.986423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:40.007670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d1331-1f2e-4223-bb71-069330f7933f None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:40.007979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d1331-1f2e-4223-bb71-069330f7933f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:40.008262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d1331-1f2e-4223-bb71-069330f7933f None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:40.009858 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 899/1755] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jul 12 13:42:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:42:40.020320 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19be6b6-1b46-4e18-b28a-7f7b8ca5f0ee None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:40.020454 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19be6b6-1b46-4e18-b28a-7f7b8ca5f0ee None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:40.020939 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19be6b6-1b46-4e18-b28a-7f7b8ca5f0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:40.030076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 857/1756] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:42:39 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:40.038736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01635075-8e95-4c77-96f8-6ff9ca21b502 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:40.058282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d19be6b6-1b46-4e18-b28a-7f7b8ca5f0ee None None] MFA Rules not processed for user `0d3915cde0d04bcfb9b5ce8c89af1e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:40.059908 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d19be6b6-1b46-4e18-b28a-7f7b8ca5f0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:40.224187 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01635075-8e95-4c77-96f8-6ff9ca21b502 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:40.236189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 900/1757] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jul 12 13:42:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:40.239791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f19784-902d-406b-a9f3-cf05458884d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:40.240736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f19784-902d-406b-a9f3-cf05458884d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:40.241171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f19784-902d-406b-a9f3-cf05458884d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:40.241616 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f19784-902d-406b-a9f3-cf05458884d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:40.242527 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f19784-902d-406b-a9f3-cf05458884d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:40.244742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f19784-902d-406b-a9f3-cf05458884d0 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:40.272640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 858/1758] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:42:40 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:40.276914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19be6b6-1b46-4e18-b28a-7f7b8ca5f0ee None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:40.293990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19be6b6-1b46-4e18-b28a-7f7b8ca5f0ee None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:40.299887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0b67f-d0ff-4700-b134-2b97ed42266b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:40.300655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0b67f-d0ff-4700-b134-2b97ed42266b admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:40.301007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0b67f-d0ff-4700-b134-2b97ed42266b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:40.301398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0b67f-d0ff-4700-b134-2b97ed42266b admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:40.302238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0b67f-d0ff-4700-b134-2b97ed42266b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:40.304433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0b67f-d0ff-4700-b134-2b97ed42266b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:40.317772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 901/1759] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:40.324157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f19784-902d-406b-a9f3-cf05458884d0 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:40.342014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f19784-902d-406b-a9f3-cf05458884d0 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:40.347416 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8aecf3-c354-4855-974a-5ec628b41747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:40.348118 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8aecf3-c354-4855-974a-5ec628b41747 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:40.348466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8aecf3-c354-4855-974a-5ec628b41747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:40.348672 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8aecf3-c354-4855-974a-5ec628b41747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:40.350504 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8aecf3-c354-4855-974a-5ec628b41747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:40.352477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8aecf3-c354-4855-974a-5ec628b41747 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:40.365604 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 859/1760] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:40.369680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0b67f-d0ff-4700-b134-2b97ed42266b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:40.395924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0b67f-d0ff-4700-b134-2b97ed42266b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:40.406380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948162ac-8518-4160-816a-e67ce77dc722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:40.408265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948162ac-8518-4160-816a-e67ce77dc722 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:40.408957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948162ac-8518-4160-816a-e67ce77dc722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:40.409466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948162ac-8518-4160-816a-e67ce77dc722 admin admin] PATH_INFO: `/v3/projects/0f1ddf63a74946f1b70dd86a49a1d4f6/users/51a980e9488c49e8b0eee114251fa496/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:40.448805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948162ac-8518-4160-816a-e67ce77dc722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f1ddf63a74946f1b70dd86a49a1d4f6, user_id=51a980e9488c49e8b0eee114251fa496, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:40.451999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948162ac-8518-4160-816a-e67ce77dc722 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:40.488432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8aecf3-c354-4855-974a-5ec628b41747 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:40.491353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0d8aecf3-c354-4855-974a-5ec628b41747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:40.519692 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 902/1761] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:42:40 2022] PUT /identity/v3/projects/0f1ddf63a74946f1b70dd86a49a1d4f6/users/51a980e9488c49e8b0eee114251fa496/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:42:40.532127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad478a-3fd7-4b26-8c96-07ce23330753 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:40.532750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad478a-3fd7-4b26-8c96-07ce23330753 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:40.533036 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad478a-3fd7-4b26-8c96-07ce23330753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:40.545171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8aecf3-c354-4855-974a-5ec628b41747 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:40.627182 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-6dad478a-3fd7-4b26-8c96-07ce23330753 None None] MFA Rules not processed for user `51a980e9488c49e8b0eee114251fa496`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:40.629370 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6dad478a-3fd7-4b26-8c96-07ce23330753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:40.659488 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9955b5e-90b3-40b4-aaa7-dded4f62461d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:40.660571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9955b5e-90b3-40b4-aaa7-dded4f62461d service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:40.660571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9955b5e-90b3-40b4-aaa7-dded4f62461d service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:40.660835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9955b5e-90b3-40b4-aaa7-dded4f62461d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:40.673039 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9955b5e-90b3-40b4-aaa7-dded4f62461d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:40.675190 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9955b5e-90b3-40b4-aaa7-dded4f62461d service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:40.705689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 860/1762] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:42:40.739241 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 903/1763] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:40.744097 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9955b5e-90b3-40b4-aaa7-dded4f62461d service nova] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:40.844900 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:40.867487 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9955b5e-90b3-40b4-aaa7-dded4f62461d service nova] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:40.883147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2470d96-97bf-4367-b56d-a76c3fdf0cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:40.884467 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2470d96-97bf-4367-b56d-a76c3fdf0cb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:40.884759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2470d96-97bf-4367-b56d-a76c3fdf0cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:40.885039 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2470d96-97bf-4367-b56d-a76c3fdf0cb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:40.885795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2470d96-97bf-4367-b56d-a76c3fdf0cb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:40.888641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2470d96-97bf-4367-b56d-a76c3fdf0cb9 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:40.956238 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 861/1764] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:42:40 2022] POST /identity/v3/projects => generated 385 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:40.961013 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad478a-3fd7-4b26-8c96-07ce23330753 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:40.967815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:40.976790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad478a-3fd7-4b26-8c96-07ce23330753 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:41.105083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eca118-db2f-41fa-953a-ef3b98a7134e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:41.105997 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eca118-db2f-41fa-953a-ef3b98a7134e admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:41.106499 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eca118-db2f-41fa-953a-ef3b98a7134e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:41.106871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eca118-db2f-41fa-953a-ef3b98a7134e admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:41.107943 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eca118-db2f-41fa-953a-ef3b98a7134e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:41.114454 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eca118-db2f-41fa-953a-ef3b98a7134e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:41.173600 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 904/1765] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:42:40 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:41.186027 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2470d96-97bf-4367-b56d-a76c3fdf0cb9 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:41.213390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2470d96-97bf-4367-b56d-a76c3fdf0cb9 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:41.220014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934435a-eb55-4cf3-b53c-db8810aea654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:41.220687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934435a-eb55-4cf3-b53c-db8810aea654 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:41.220970 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934435a-eb55-4cf3-b53c-db8810aea654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:41.221273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934435a-eb55-4cf3-b53c-db8810aea654 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:41.225220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934435a-eb55-4cf3-b53c-db8810aea654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:41.226869 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934435a-eb55-4cf3-b53c-db8810aea654 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:41.242235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 862/1766] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:41.246098 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eca118-db2f-41fa-953a-ef3b98a7134e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:41.260664 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eca118-db2f-41fa-953a-ef3b98a7134e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:41.267763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52d782-e0e5-4efb-a80b-3871c0f6ef08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:41.269108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52d782-e0e5-4efb-a80b-3871c0f6ef08 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:41.269497 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52d782-e0e5-4efb-a80b-3871c0f6ef08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:41.269922 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52d782-e0e5-4efb-a80b-3871c0f6ef08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:41.270806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52d782-e0e5-4efb-a80b-3871c0f6ef08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:41.273599 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52d782-e0e5-4efb-a80b-3871c0f6ef08 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:41.289385 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 905/1767] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:42:41.294272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934435a-eb55-4cf3-b53c-db8810aea654 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:41.309370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934435a-eb55-4cf3-b53c-db8810aea654 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:41.315620 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564330d-249e-4ff2-8b11-d459a28e7563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:41.316626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564330d-249e-4ff2-8b11-d459a28e7563 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:41.317118 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564330d-249e-4ff2-8b11-d459a28e7563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:41.317535 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564330d-249e-4ff2-8b11-d459a28e7563 admin admin] PATH_INFO: `/v3/projects/ffb30ded7afa402d8e10f66b50a144ec/users/30ed3f84994448979d6d186202f47200/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:41.345014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564330d-249e-4ff2-8b11-d459a28e7563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb30ded7afa402d8e10f66b50a144ec, user_id=30ed3f84994448979d6d186202f47200, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:41.351499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564330d-249e-4ff2-8b11-d459a28e7563 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:41.377020 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 863/1768] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:42:41 2022] PUT /identity/v3/projects/ffb30ded7afa402d8e10f66b50a144ec/users/30ed3f84994448979d6d186202f47200/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:42:41.389002 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e679fb-0cc1-4783-adec-3488fb8104ad None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:41.389446 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e679fb-0cc1-4783-adec-3488fb8104ad None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:41.389753 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e679fb-0cc1-4783-adec-3488fb8104ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:41.459949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-85e679fb-0cc1-4783-adec-3488fb8104ad None None] MFA Rules not processed for user `30ed3f84994448979d6d186202f47200`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:41.465061 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-85e679fb-0cc1-4783-adec-3488fb8104ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:41.676228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 906/1769] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:41.681357 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564330d-249e-4ff2-8b11-d459a28e7563 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:41.840505 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564330d-249e-4ff2-8b11-d459a28e7563 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:41.853041 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e450e-8325-4ee9-9306-acaff789378f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:41.854544 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e450e-8325-4ee9-9306-acaff789378f admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:41.855482 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e450e-8325-4ee9-9306-acaff789378f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:41.856080 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e450e-8325-4ee9-9306-acaff789378f admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:41.857774 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e450e-8325-4ee9-9306-acaff789378f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:41.861317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e450e-8325-4ee9-9306-acaff789378f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:41.903193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 864/1770] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:42:41 2022] POST /identity/v3/projects => generated 387 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:41.907012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e679fb-0cc1-4783-adec-3488fb8104ad None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:41.926685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e679fb-0cc1-4783-adec-3488fb8104ad None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:42.024256 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d6c7a-5e9c-4173-9c9e-f54840f16973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:42.025832 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d6c7a-5e9c-4173-9c9e-f54840f16973 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:42.026171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d6c7a-5e9c-4173-9c9e-f54840f16973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:42.026450 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d6c7a-5e9c-4173-9c9e-f54840f16973 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:42.027549 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d6c7a-5e9c-4173-9c9e-f54840f16973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:42.031660 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d6c7a-5e9c-4173-9c9e-f54840f16973 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:42.082342 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:42.090268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 907/1771] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:42:41 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:42.094686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e450e-8325-4ee9-9306-acaff789378f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:42.110271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e450e-8325-4ee9-9306-acaff789378f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:42.125490 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5cd37-2efe-4fc4-9f5c-576cd5583b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:42.126876 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5cd37-2efe-4fc4-9f5c-576cd5583b71 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:42.127225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5cd37-2efe-4fc4-9f5c-576cd5583b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:42.127610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5cd37-2efe-4fc4-9f5c-576cd5583b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:42.128422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5cd37-2efe-4fc4-9f5c-576cd5583b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:42.130383 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5cd37-2efe-4fc4-9f5c-576cd5583b71 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:42.166745 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 865/1772] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:42.171006 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d6c7a-5e9c-4173-9c9e-f54840f16973 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:42.191808 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d6c7a-5e9c-4173-9c9e-f54840f16973 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:42.209237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33df226-1e09-4140-8309-009ab023e040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:42.210994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33df226-1e09-4140-8309-009ab023e040 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:42.211427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33df226-1e09-4140-8309-009ab023e040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:42.212613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33df226-1e09-4140-8309-009ab023e040 admin admin] PATH_INFO: `/v3/projects/f2233b85afde4af690fb8f474c4e459f/users/f49df70c726340f784dc7d75bb141cf4/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:42.257048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33df226-1e09-4140-8309-009ab023e040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2233b85afde4af690fb8f474c4e459f, user_id=f49df70c726340f784dc7d75bb141cf4, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:42.262589 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33df226-1e09-4140-8309-009ab023e040 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:42.299040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 908/1773] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:42:42 2022] PUT /identity/v3/projects/f2233b85afde4af690fb8f474c4e459f/users/f49df70c726340f784dc7d75bb141cf4/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:42.305993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5cd37-2efe-4fc4-9f5c-576cd5583b71 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:42.331544 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5cd37-2efe-4fc4-9f5c-576cd5583b71 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:42.539458 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b62434-eabb-48ad-b2f5-e9f5679491da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:42.540239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b62434-eabb-48ad-b2f5-e9f5679491da admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:42.540457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b62434-eabb-48ad-b2f5-e9f5679491da admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:42.540717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b62434-eabb-48ad-b2f5-e9f5679491da admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:42.541459 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b62434-eabb-48ad-b2f5-e9f5679491da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:42.543184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b62434-eabb-48ad-b2f5-e9f5679491da admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:42.565083 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 866/1774] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:42.569770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33df226-1e09-4140-8309-009ab023e040 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:42.620511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33df226-1e09-4140-8309-009ab023e040 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:42.628757 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def386d-e254-4d8d-bfbe-431a6d786867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:42.629687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def386d-e254-4d8d-bfbe-431a6d786867 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:42.630053 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def386d-e254-4d8d-bfbe-431a6d786867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:42.630341 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def386d-e254-4d8d-bfbe-431a6d786867 admin admin] PATH_INFO: `/v3/projects/f2233b85afde4af690fb8f474c4e459f/users/f49df70c726340f784dc7d75bb141cf4/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:42.632480 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def386d-e254-4d8d-bfbe-431a6d786867 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2233b85afde4af690fb8f474c4e459f, user_id=f49df70c726340f784dc7d75bb141cf4, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:42.636608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def386d-e254-4d8d-bfbe-431a6d786867 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:42.667262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 909/1775] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:42:42 2022] PUT /identity/v3/projects/f2233b85afde4af690fb8f474c4e459f/users/f49df70c726340f784dc7d75bb141cf4/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:42.678969 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a831dc-dc55-40af-84a6-ff8ec3e99271 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:42.679380 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a831dc-dc55-40af-84a6-ff8ec3e99271 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:42.680040 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a831dc-dc55-40af-84a6-ff8ec3e99271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:42.725385 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-20a831dc-dc55-40af-84a6-ff8ec3e99271 None None] MFA Rules not processed for user `f49df70c726340f784dc7d75bb141cf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:42.727141 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-20a831dc-dc55-40af-84a6-ff8ec3e99271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:42.742251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:42.914176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 867/1776] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:42.924064 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def386d-e254-4d8d-bfbe-431a6d786867 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:42.927842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2def386d-e254-4d8d-bfbe-431a6d786867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:43.028960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def386d-e254-4d8d-bfbe-431a6d786867 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:43.035164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed5745-b115-49f9-b563-0f9fc7a7e38b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:43.036624 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed5745-b115-49f9-b563-0f9fc7a7e38b service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:43.037097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed5745-b115-49f9-b563-0f9fc7a7e38b service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:43.037990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed5745-b115-49f9-b563-0f9fc7a7e38b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:43.052243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed5745-b115-49f9-b563-0f9fc7a7e38b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:43.060921 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed5745-b115-49f9-b563-0f9fc7a7e38b service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:43.092556 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 910/1777] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:43.784086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a831dc-dc55-40af-84a6-ff8ec3e99271 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:43.813506 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a831dc-dc55-40af-84a6-ff8ec3e99271 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:43.824993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4f32e-34b1-469f-ac87-348c915adda2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:43.827151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4f32e-34b1-469f-ac87-348c915adda2 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:43.827151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4f32e-34b1-469f-ac87-348c915adda2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:43.827151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4f32e-34b1-469f-ac87-348c915adda2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:43.921163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4f32e-34b1-469f-ac87-348c915adda2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:43.922723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4f32e-34b1-469f-ac87-348c915adda2 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:43.938315 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 868/1778] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:49.131362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed5745-b115-49f9-b563-0f9fc7a7e38b service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:49.132808 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-24ed5745-b115-49f9-b563-0f9fc7a7e38b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:49.201916 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed5745-b115-49f9-b563-0f9fc7a7e38b service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:49.207100 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ab223-9d7d-4abc-9653-867e930c38b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:49.208173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ab223-9d7d-4abc-9653-867e930c38b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:49.208489 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ab223-9d7d-4abc-9653-867e930c38b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:49.208764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ab223-9d7d-4abc-9653-867e930c38b4 admin admin] PATH_INFO: `/v3/users/2f7c0ccd5cea4dfb80d041d44c835a5e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:49.210005 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ab223-9d7d-4abc-9653-867e930c38b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f7c0ccd5cea4dfb80d041d44c835a5e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:49.211970 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ab223-9d7d-4abc-9653-867e930c38b4 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:49.279033 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-637ab223-9d7d-4abc-9653-867e930c38b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f7c0ccd5cea4dfb80d041d44c835a5e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:49.279747 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-637ab223-9d7d-4abc-9653-867e930c38b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f7c0ccd5cea4dfb80d041d44c835a5e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:49.280557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-637ab223-9d7d-4abc-9653-867e930c38b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f7c0ccd5cea4dfb80d041d44c835a5e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:49.301939 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-637ab223-9d7d-4abc-9653-867e930c38b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f7c0ccd5cea4dfb80d041d44c835a5e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:49.305605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:49.317591 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-637ab223-9d7d-4abc-9653-867e930c38b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f7c0ccd5cea4dfb80d041d44c835a5e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:49.336288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 911/1779] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:42:49 2022] DELETE /identity/v3/users/2f7c0ccd5cea4dfb80d041d44c835a5e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:49.340816 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ce55a-0489-4391-8bb1-ad67b8b0d4e9 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:49.341281 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ce55a-0489-4391-8bb1-ad67b8b0d4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:49.341355 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ce55a-0489-4391-8bb1-ad67b8b0d4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:49.373703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0c5ce55a-0489-4391-8bb1-ad67b8b0d4e9 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:49.373703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0c5ce55a-0489-4391-8bb1-ad67b8b0d4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:49.447944 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 869/1780] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:49.451664 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ab223-9d7d-4abc-9653-867e930c38b4 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:49.457047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-637ab223-9d7d-4abc-9653-867e930c38b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:49.462319 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ce55a-0489-4391-8bb1-ad67b8b0d4e9 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:49.466723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0c5ce55a-0489-4391-8bb1-ad67b8b0d4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:49.546622 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ab223-9d7d-4abc-9653-867e930c38b4 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:49.550619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96aac4-79fc-4ae8-8e52-a5e3f931e2b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:49.551343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96aac4-79fc-4ae8-8e52-a5e3f931e2b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:49.551711 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96aac4-79fc-4ae8-8e52-a5e3f931e2b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:49.552063 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96aac4-79fc-4ae8-8e52-a5e3f931e2b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:49.554205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ec96aac4-79fc-4ae8-8e52-a5e3f931e2b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:49.560260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:49.576950 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ce55a-0489-4391-8bb1-ad67b8b0d4e9 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:49.582729 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d2866-0635-4c6d-9927-3d134a31aa5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:49.584110 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d2866-0635-4c6d-9927-3d134a31aa5e service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:49.584443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d2866-0635-4c6d-9927-3d134a31aa5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:49.584720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d2866-0635-4c6d-9927-3d134a31aa5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:49.600076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d2866-0635-4c6d-9927-3d134a31aa5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:49.601583 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d2866-0635-4c6d-9927-3d134a31aa5e service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:49.615979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 870/1781] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:49.669845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96aac4-79fc-4ae8-8e52-a5e3f931e2b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:49.671452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96aac4-79fc-4ae8-8e52-a5e3f931e2b4 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:49.687195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 912/1782] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:50.208784 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d2866-0635-4c6d-9927-3d134a31aa5e service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:50.212543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-088d2866-0635-4c6d-9927-3d134a31aa5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:50.234324 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d2866-0635-4c6d-9927-3d134a31aa5e service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:50.241404 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5db9c-f8da-44b9-891a-307cfad6bd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:50.242662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5db9c-f8da-44b9-891a-307cfad6bd71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:50.242957 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5db9c-f8da-44b9-891a-307cfad6bd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:50.243263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5db9c-f8da-44b9-891a-307cfad6bd71 admin admin] PATH_INFO: `/v3/projects/9e4da71d5a8f4c7c8aa9a4457f984a53` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:50.244736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5db9c-f8da-44b9-891a-307cfad6bd71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e4da71d5a8f4c7c8aa9a4457f984a53)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:50.246735 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5db9c-f8da-44b9-891a-307cfad6bd71 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:50.331657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f1e5db9c-f8da-44b9-891a-307cfad6bd71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e4da71d5a8f4c7c8aa9a4457f984a53', 'request_id': 'req-f1e5db9c-f8da-44b9-891a-307cfad6bd71'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:50.339332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f1e5db9c-f8da-44b9-891a-307cfad6bd71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e4da71d5a8f4c7c8aa9a4457f984a53', 'request_id': 'req-f1e5db9c-f8da-44b9-891a-307cfad6bd71'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:50.357684 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f1e5db9c-f8da-44b9-891a-307cfad6bd71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e4da71d5a8f4c7c8aa9a4457f984a53', 'request_id': 'req-f1e5db9c-f8da-44b9-891a-307cfad6bd71'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:50.377156 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f1e5db9c-f8da-44b9-891a-307cfad6bd71 admin admin] The token cache is being invalidate because project 9e4da71d5a8f4c7c8aa9a4457f984a53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:42:50.377713 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f1e5db9c-f8da-44b9-891a-307cfad6bd71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:50.379296 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f1e5db9c-f8da-44b9-891a-307cfad6bd71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:50.384791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 871/1783] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:42:50 2022] DELETE /identity/v3/projects/9e4da71d5a8f4c7c8aa9a4457f984a53 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:50.386173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96aac4-79fc-4ae8-8e52-a5e3f931e2b4 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:50.399978 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96aac4-79fc-4ae8-8e52-a5e3f931e2b4 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:50.523121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c7d9c-4a9e-4bea-b1ad-705582af7371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:50.524086 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c7d9c-4a9e-4bea-b1ad-705582af7371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:50.524350 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c7d9c-4a9e-4bea-b1ad-705582af7371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:50.524668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c7d9c-4a9e-4bea-b1ad-705582af7371 admin admin] PATH_INFO: `/v3/users/edb18b1a5e14429dbafb543e3d3bb617` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:50.530340 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4c7d9c-4a9e-4bea-b1ad-705582af7371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edb18b1a5e14429dbafb543e3d3bb617)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:50.533133 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4c7d9c-4a9e-4bea-b1ad-705582af7371 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:50.615312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:50.626783 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2b4c7d9c-4a9e-4bea-b1ad-705582af7371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edb18b1a5e14429dbafb543e3d3bb617'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:50.630667 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2b4c7d9c-4a9e-4bea-b1ad-705582af7371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edb18b1a5e14429dbafb543e3d3bb617'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:50.631639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2b4c7d9c-4a9e-4bea-b1ad-705582af7371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edb18b1a5e14429dbafb543e3d3bb617'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:50.661249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2b4c7d9c-4a9e-4bea-b1ad-705582af7371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edb18b1a5e14429dbafb543e3d3bb617'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:50.669088 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2b4c7d9c-4a9e-4bea-b1ad-705582af7371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edb18b1a5e14429dbafb543e3d3bb617'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:50.703361 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 913/1784] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:42:50 2022] DELETE /identity/v3/users/edb18b1a5e14429dbafb543e3d3bb617 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:51.120103 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5db9c-f8da-44b9-891a-307cfad6bd71 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:51.121628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f1e5db9c-f8da-44b9-891a-307cfad6bd71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:51.137836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5db9c-f8da-44b9-891a-307cfad6bd71 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:51.199207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5643ed2-e2a8-4d15-946f-b9414375c13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:51.200083 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5643ed2-e2a8-4d15-946f-b9414375c13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:51.200366 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5643ed2-e2a8-4d15-946f-b9414375c13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:51.200644 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5643ed2-e2a8-4d15-946f-b9414375c13f admin admin] PATH_INFO: `/v3/projects/c6fe68a410124081bd4aec3cd35c3751` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:51.201842 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5643ed2-e2a8-4d15-946f-b9414375c13f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6fe68a410124081bd4aec3cd35c3751)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:51.203785 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5643ed2-e2a8-4d15-946f-b9414375c13f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:51.250099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f5643ed2-e2a8-4d15-946f-b9414375c13f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6fe68a410124081bd4aec3cd35c3751', 'request_id': 'req-f5643ed2-e2a8-4d15-946f-b9414375c13f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:51.255784 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f5643ed2-e2a8-4d15-946f-b9414375c13f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6fe68a410124081bd4aec3cd35c3751', 'request_id': 'req-f5643ed2-e2a8-4d15-946f-b9414375c13f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:51.263840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f5643ed2-e2a8-4d15-946f-b9414375c13f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6fe68a410124081bd4aec3cd35c3751', 'request_id': 'req-f5643ed2-e2a8-4d15-946f-b9414375c13f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:51.271405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:51.276766 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f5643ed2-e2a8-4d15-946f-b9414375c13f admin admin] The token cache is being invalidate because project c6fe68a410124081bd4aec3cd35c3751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:42:51.277254 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f5643ed2-e2a8-4d15-946f-b9414375c13f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:51.278295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f5643ed2-e2a8-4d15-946f-b9414375c13f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:51.280261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 872/1785] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:42:51 2022] DELETE /identity/v3/projects/c6fe68a410124081bd4aec3cd35c3751 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:51.285145 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25719c12-33b3-481b-863b-979b5f9fa00e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:51.285441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25719c12-33b3-481b-863b-979b5f9fa00e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:51.285721 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25719c12-33b3-481b-863b-979b5f9fa00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:51.304283 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-25719c12-33b3-481b-863b-979b5f9fa00e None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:51.305611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-25719c12-33b3-481b-863b-979b5f9fa00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:51.366592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 914/1786] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:51.375074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa18a5-e9b3-435d-9306-c35ef37d5564 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:51.375550 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa18a5-e9b3-435d-9306-c35ef37d5564 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:51.375550 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa18a5-e9b3-435d-9306-c35ef37d5564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:51.395462 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-6bfa18a5-e9b3-435d-9306-c35ef37d5564 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:51.397354 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6bfa18a5-e9b3-435d-9306-c35ef37d5564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:51.405327 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 873/1787] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:51.409508 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25719c12-33b3-481b-863b-979b5f9fa00e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:51.420251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25719c12-33b3-481b-863b-979b5f9fa00e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:51.425717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341417cb-df5f-4e72-a4a7-f4f3da5c63ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:51.426420 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341417cb-df5f-4e72-a4a7-f4f3da5c63ec admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:51.426744 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341417cb-df5f-4e72-a4a7-f4f3da5c63ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:51.427034 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341417cb-df5f-4e72-a4a7-f4f3da5c63ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:51.427943 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341417cb-df5f-4e72-a4a7-f4f3da5c63ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:42:51.428442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341417cb-df5f-4e72-a4a7-f4f3da5c63ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:51.429929 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341417cb-df5f-4e72-a4a7-f4f3da5c63ec admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:51.448185 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 915/1788] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:42:51.452772 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa18a5-e9b3-435d-9306-c35ef37d5564 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:51.469908 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa18a5-e9b3-435d-9306-c35ef37d5564 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:51.478747 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9bc13-8b0b-4266-bbba-437b52503bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:51.479554 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9bc13-8b0b-4266-bbba-437b52503bae admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:51.479903 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9bc13-8b0b-4266-bbba-437b52503bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:51.480244 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9bc13-8b0b-4266-bbba-437b52503bae admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:51.481061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f9bc13-8b0b-4266-bbba-437b52503bae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:51.483248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f9bc13-8b0b-4266-bbba-437b52503bae admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:51.512932 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 874/1789] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:42:51 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:42:51.516852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341417cb-df5f-4e72-a4a7-f4f3da5c63ec admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:51.527648 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341417cb-df5f-4e72-a4a7-f4f3da5c63ec admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:51.587772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83cf53a-ec44-4db2-8bdd-4d7805355373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:51.588491 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cf53a-ec44-4db2-8bdd-4d7805355373 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:51.588777 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cf53a-ec44-4db2-8bdd-4d7805355373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:51.589060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cf53a-ec44-4db2-8bdd-4d7805355373 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:51.589916 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83cf53a-ec44-4db2-8bdd-4d7805355373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:51.591976 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83cf53a-ec44-4db2-8bdd-4d7805355373 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:51.616383 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 916/1790] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:42:51 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:42:51.620535 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9bc13-8b0b-4266-bbba-437b52503bae admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:51.631605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9bc13-8b0b-4266-bbba-437b52503bae admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:51.636215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c85d13-408d-4ab3-a0bf-7a4fbb7f2493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:51.636903 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c85d13-408d-4ab3-a0bf-7a4fbb7f2493 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:51.637184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c85d13-408d-4ab3-a0bf-7a4fbb7f2493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:51.637454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c85d13-408d-4ab3-a0bf-7a4fbb7f2493 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:51.638217 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c85d13-408d-4ab3-a0bf-7a4fbb7f2493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:51.639793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c85d13-408d-4ab3-a0bf-7a4fbb7f2493 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:51.650389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 875/1791] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:42:51.654717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83cf53a-ec44-4db2-8bdd-4d7805355373 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:51.665567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83cf53a-ec44-4db2-8bdd-4d7805355373 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:51.670279 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d3157-0fb8-4c4d-a350-11b124153892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:51.670952 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d3157-0fb8-4c4d-a350-11b124153892 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:51.671241 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d3157-0fb8-4c4d-a350-11b124153892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:51.671519 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d3157-0fb8-4c4d-a350-11b124153892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:51.672251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d3157-0fb8-4c4d-a350-11b124153892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:51.673888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d3157-0fb8-4c4d-a350-11b124153892 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:51.685151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 917/1792] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:51.689205 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c85d13-408d-4ab3-a0bf-7a4fbb7f2493 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:51.699137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c85d13-408d-4ab3-a0bf-7a4fbb7f2493 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:51.703763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a99dbc-e437-4707-ae18-58bba1ac7ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:51.704638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a99dbc-e437-4707-ae18-58bba1ac7ced admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:51.704917 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a99dbc-e437-4707-ae18-58bba1ac7ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:51.705194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a99dbc-e437-4707-ae18-58bba1ac7ced admin admin] PATH_INFO: `/v3/projects/c1efcc2c519f4699a67aefc2edad6593/users/000408e897c644c6bc13854132d2b5b3/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:51.723591 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a99dbc-e437-4707-ae18-58bba1ac7ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1efcc2c519f4699a67aefc2edad6593, user_id=000408e897c644c6bc13854132d2b5b3, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:51.727392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a99dbc-e437-4707-ae18-58bba1ac7ced admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:51.743297 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 876/1793] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:42:51 2022] PUT /identity/v3/projects/c1efcc2c519f4699a67aefc2edad6593/users/000408e897c644c6bc13854132d2b5b3/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:51.754106 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda9e0c-27f8-42d2-b5e1-b3d85f5a6629 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:51.755847 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda9e0c-27f8-42d2-b5e1-b3d85f5a6629 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:51.755847 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda9e0c-27f8-42d2-b5e1-b3d85f5a6629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:51.789498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-afda9e0c-27f8-42d2-b5e1-b3d85f5a6629 None None] MFA Rules not processed for user `000408e897c644c6bc13854132d2b5b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:51.789498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-afda9e0c-27f8-42d2-b5e1-b3d85f5a6629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:51.801166 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:51.918795 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 918/1794] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:51 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:51.919843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a99dbc-e437-4707-ae18-58bba1ac7ced admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:52.109884 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a99dbc-e437-4707-ae18-58bba1ac7ced admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:52.115610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ac754-d401-4fa3-919d-9d60a14c3cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:52.123392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ac754-d401-4fa3-919d-9d60a14c3cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:52.123707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ac754-d401-4fa3-919d-9d60a14c3cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:52.123980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ac754-d401-4fa3-919d-9d60a14c3cd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:52.124787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ac754-d401-4fa3-919d-9d60a14c3cd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:52.127368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ac754-d401-4fa3-919d-9d60a14c3cd6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:52.196896 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 877/1795] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:42:51 2022] POST /identity/v3/projects => generated 363 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:52.200369 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda9e0c-27f8-42d2-b5e1-b3d85f5a6629 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:52.222403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda9e0c-27f8-42d2-b5e1-b3d85f5a6629 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:52.388490 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ba5f5-956f-4bb7-b7e4-59310af5e0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:52.389391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ba5f5-956f-4bb7-b7e4-59310af5e0dc admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:52.390310 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ba5f5-956f-4bb7-b7e4-59310af5e0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:52.390310 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ba5f5-956f-4bb7-b7e4-59310af5e0dc admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:52.391950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ba5f5-956f-4bb7-b7e4-59310af5e0dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:52.396410 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ba5f5-956f-4bb7-b7e4-59310af5e0dc admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:52.442598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 919/1796] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:42:52 2022] POST /identity/v3/users => generated 288 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:52.452543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ac754-d401-4fa3-919d-9d60a14c3cd6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:52.471172 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ac754-d401-4fa3-919d-9d60a14c3cd6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:52.486716 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe8015-8bdb-499d-aa03-58bc31606751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:52.487400 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe8015-8bdb-499d-aa03-58bc31606751 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:52.487861 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe8015-8bdb-499d-aa03-58bc31606751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:52.488415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe8015-8bdb-499d-aa03-58bc31606751 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:52.489489 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe8015-8bdb-499d-aa03-58bc31606751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:52.504091 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe8015-8bdb-499d-aa03-58bc31606751 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:52.526352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 878/1797] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:52.527981 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ba5f5-956f-4bb7-b7e4-59310af5e0dc admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:52.544808 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ba5f5-956f-4bb7-b7e4-59310af5e0dc admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:52.576576 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140a48e-d452-4ee0-bdef-9bd0de6dab43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:52.577520 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140a48e-d452-4ee0-bdef-9bd0de6dab43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:52.577788 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140a48e-d452-4ee0-bdef-9bd0de6dab43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:52.578097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140a48e-d452-4ee0-bdef-9bd0de6dab43 admin admin] PATH_INFO: `/v3/projects/1690e8e04a3b48ebbe99c45cbe8a28a2/users/9d9f9fb81ae547f79563d0090188704c/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:52.621646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe8015-8bdb-499d-aa03-58bc31606751 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:52.624917 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-46fe8015-8bdb-499d-aa03-58bc31606751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:52.627889 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140a48e-d452-4ee0-bdef-9bd0de6dab43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1690e8e04a3b48ebbe99c45cbe8a28a2, user_id=9d9f9fb81ae547f79563d0090188704c, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:52.631387 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140a48e-d452-4ee0-bdef-9bd0de6dab43 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:52.641407 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe8015-8bdb-499d-aa03-58bc31606751 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:52.657525 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf982b-2639-4fbe-a3c8-16a967ecf745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:52.661423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf982b-2639-4fbe-a3c8-16a967ecf745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:52.661844 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf982b-2639-4fbe-a3c8-16a967ecf745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:52.663888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf982b-2639-4fbe-a3c8-16a967ecf745 admin admin] PATH_INFO: `/v3/users/51a980e9488c49e8b0eee114251fa496` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:52.670043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 920/1798] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:42:52 2022] PUT /identity/v3/projects/1690e8e04a3b48ebbe99c45cbe8a28a2/users/9d9f9fb81ae547f79563d0090188704c/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:42:52.670633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf982b-2639-4fbe-a3c8-16a967ecf745 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51a980e9488c49e8b0eee114251fa496)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:52.670633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf982b-2639-4fbe-a3c8-16a967ecf745 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:52.672118 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140a48e-d452-4ee0-bdef-9bd0de6dab43 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:52.766488 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9ebf982b-2639-4fbe-a3c8-16a967ecf745 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51a980e9488c49e8b0eee114251fa496'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:52.787476 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:52.790641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9ebf982b-2639-4fbe-a3c8-16a967ecf745 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51a980e9488c49e8b0eee114251fa496'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:52.808430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9ebf982b-2639-4fbe-a3c8-16a967ecf745 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51a980e9488c49e8b0eee114251fa496'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:52.808995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140a48e-d452-4ee0-bdef-9bd0de6dab43 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:52.810012 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9ebf982b-2639-4fbe-a3c8-16a967ecf745 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51a980e9488c49e8b0eee114251fa496'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:52.821283 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae27c4-b4c7-4a8d-84d1-96676291e608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:52.822099 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae27c4-b4c7-4a8d-84d1-96676291e608 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:52.822392 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae27c4-b4c7-4a8d-84d1-96676291e608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:52.822678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae27c4-b4c7-4a8d-84d1-96676291e608 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:52.823529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae27c4-b4c7-4a8d-84d1-96676291e608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:52.825272 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae27c4-b4c7-4a8d-84d1-96676291e608 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:52.833224 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9ebf982b-2639-4fbe-a3c8-16a967ecf745 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51a980e9488c49e8b0eee114251fa496'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:52.846004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 879/1799] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:42:52 2022] DELETE /identity/v3/users/51a980e9488c49e8b0eee114251fa496 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:52.853592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006766c-7bdb-40a2-87c6-94d59bf3d7d5 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:52.854153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006766c-7bdb-40a2-87c6-94d59bf3d7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:52.854712 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006766c-7bdb-40a2-87c6-94d59bf3d7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:52.883521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 921/1800] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:42:52.884683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae27c4-b4c7-4a8d-84d1-96676291e608 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:52.887654 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d1ae27c4-b4c7-4a8d-84d1-96676291e608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:52.889705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0006766c-7bdb-40a2-87c6-94d59bf3d7d5 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:52.891291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0006766c-7bdb-40a2-87c6-94d59bf3d7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:52.916135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae27c4-b4c7-4a8d-84d1-96676291e608 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:52.938921 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:53.027349 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e9f2d-9884-4c90-84c7-afdd2cbddd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:53.028338 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e9f2d-9884-4c90-84c7-afdd2cbddd3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:53.028649 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e9f2d-9884-4c90-84c7-afdd2cbddd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:53.029151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e9f2d-9884-4c90-84c7-afdd2cbddd3c admin admin] PATH_INFO: `/v3/projects/1690e8e04a3b48ebbe99c45cbe8a28a2/users/9d9f9fb81ae547f79563d0090188704c/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:53.032093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135e9f2d-9884-4c90-84c7-afdd2cbddd3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1690e8e04a3b48ebbe99c45cbe8a28a2, user_id=9d9f9fb81ae547f79563d0090188704c, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:53.036266 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135e9f2d-9884-4c90-84c7-afdd2cbddd3c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:53.054237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 922/1801] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:42:52 2022] PUT /identity/v3/projects/1690e8e04a3b48ebbe99c45cbe8a28a2/users/9d9f9fb81ae547f79563d0090188704c/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:53.064704 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f857bc-f2d5-4cd8-b5c8-7c74dfee9026 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:53.065032 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f857bc-f2d5-4cd8-b5c8-7c74dfee9026 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:53.065284 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f857bc-f2d5-4cd8-b5c8-7c74dfee9026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:53.088113 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 880/1802] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:42:53.108098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006766c-7bdb-40a2-87c6-94d59bf3d7d5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:53.109883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0006766c-7bdb-40a2-87c6-94d59bf3d7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:53.113335 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-38f857bc-f2d5-4cd8-b5c8-7c74dfee9026 None None] MFA Rules not processed for user `9d9f9fb81ae547f79563d0090188704c`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:53.114810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-38f857bc-f2d5-4cd8-b5c8-7c74dfee9026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:53.185221 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006766c-7bdb-40a2-87c6-94d59bf3d7d5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:53.212787 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 923/1803] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:53 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:53.228324 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f857bc-f2d5-4cd8-b5c8-7c74dfee9026 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:53.230061 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-38f857bc-f2d5-4cd8-b5c8-7c74dfee9026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:53.250511 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81329c-0b5b-4622-8fc6-0eb7b6b4bdff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:53.251190 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81329c-0b5b-4622-8fc6-0eb7b6b4bdff service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:53.251732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81329c-0b5b-4622-8fc6-0eb7b6b4bdff service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:53.252013 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81329c-0b5b-4622-8fc6-0eb7b6b4bdff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:53.302716 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f857bc-f2d5-4cd8-b5c8-7c74dfee9026 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:53.306160 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:53.307647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcb810-e5a7-4f1a-a248-123ceda2c6d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:53.308562 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcb810-e5a7-4f1a-a248-123ceda2c6d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:53.308856 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcb810-e5a7-4f1a-a248-123ceda2c6d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:53.309138 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcb810-e5a7-4f1a-a248-123ceda2c6d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:53.323311 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81329c-0b5b-4622-8fc6-0eb7b6b4bdff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:53.324621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81329c-0b5b-4622-8fc6-0eb7b6b4bdff service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:53.326477 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcb810-e5a7-4f1a-a248-123ceda2c6d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:53.327978 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcb810-e5a7-4f1a-a248-123ceda2c6d7 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:53.335725 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 881/1804] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:42:53.340179 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 924/1805] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:53 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:42:53.415550 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81329c-0b5b-4622-8fc6-0eb7b6b4bdff service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:53.424970 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81329c-0b5b-4622-8fc6-0eb7b6b4bdff service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:53.429194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6cd45-15b9-46cc-b5af-372f9545b8c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:53.429835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6cd45-15b9-46cc-b5af-372f9545b8c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:53.430153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6cd45-15b9-46cc-b5af-372f9545b8c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:53.430427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6cd45-15b9-46cc-b5af-372f9545b8c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:53.432760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9fe6cd45-15b9-46cc-b5af-372f9545b8c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:53.552480 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6cd45-15b9-46cc-b5af-372f9545b8c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:53.553941 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6cd45-15b9-46cc-b5af-372f9545b8c9 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:53.575183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 882/1806] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:53 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:53.750529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcb810-e5a7-4f1a-a248-123ceda2c6d7 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:53.752290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-41dcb810-e5a7-4f1a-a248-123ceda2c6d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:53.784115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcb810-e5a7-4f1a-a248-123ceda2c6d7 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:53.792045 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:53.793281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:53.793573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:53.794148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56 admin admin] PATH_INFO: `/v3/projects/0f1ddf63a74946f1b70dd86a49a1d4f6` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:53.795256 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f1ddf63a74946f1b70dd86a49a1d4f6)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:53.797228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:53.841106 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:53.882603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f1ddf63a74946f1b70dd86a49a1d4f6', 'request_id': 'req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:53.905317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f1ddf63a74946f1b70dd86a49a1d4f6', 'request_id': 'req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:53.917631 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f1ddf63a74946f1b70dd86a49a1d4f6', 'request_id': 'req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:53.940028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56 admin admin] The token cache is being invalidate because project 0f1ddf63a74946f1b70dd86a49a1d4f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:42:53.940681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:53.941736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:53.944165 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 925/1807] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:42:53 2022] DELETE /identity/v3/projects/0f1ddf63a74946f1b70dd86a49a1d4f6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:53.948657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6cd45-15b9-46cc-b5af-372f9545b8c9 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:53.964959 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6cd45-15b9-46cc-b5af-372f9545b8c9 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:54.032866 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7b55f-4cc1-40ca-a360-932f20e0e75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:54.033746 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7b55f-4cc1-40ca-a360-932f20e0e75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:54.034132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7b55f-4cc1-40ca-a360-932f20e0e75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:54.034343 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7b55f-4cc1-40ca-a360-932f20e0e75a admin admin] PATH_INFO: `/v3/users/30ed3f84994448979d6d186202f47200` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:54.035565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7b55f-4cc1-40ca-a360-932f20e0e75a admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ed3f84994448979d6d186202f47200)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:54.037552 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7b55f-4cc1-40ca-a360-932f20e0e75a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:54.092074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-bcc7b55f-4cc1-40ca-a360-932f20e0e75a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ed3f84994448979d6d186202f47200'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:54.103927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-bcc7b55f-4cc1-40ca-a360-932f20e0e75a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ed3f84994448979d6d186202f47200'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:54.111572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-bcc7b55f-4cc1-40ca-a360-932f20e0e75a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ed3f84994448979d6d186202f47200'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:54.112266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-bcc7b55f-4cc1-40ca-a360-932f20e0e75a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ed3f84994448979d6d186202f47200'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:54.127574 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-bcc7b55f-4cc1-40ca-a360-932f20e0e75a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ed3f84994448979d6d186202f47200'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:54.140191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 883/1808] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:42:53 2022] DELETE /identity/v3/users/30ed3f84994448979d6d186202f47200 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:54.515197 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:54.516732 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:54.530134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0e7ef-2a5f-424e-abca-25b1eae67b56 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:54.592726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:54.593637 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:54.593930 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:54.594241 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea admin admin] PATH_INFO: `/v3/projects/ffb30ded7afa402d8e10f66b50a144ec` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:54.595456 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffb30ded7afa402d8e10f66b50a144ec)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:54.597411 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:54.646273 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffb30ded7afa402d8e10f66b50a144ec', 'request_id': 'req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:54.653802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffb30ded7afa402d8e10f66b50a144ec', 'request_id': 'req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:54.660102 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffb30ded7afa402d8e10f66b50a144ec', 'request_id': 'req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:54.671680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea admin admin] The token cache is being invalidate because project ffb30ded7afa402d8e10f66b50a144ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:42:54.671969 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:54.672635 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:54.674807 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 926/1809] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:42:54 2022] DELETE /identity/v3/projects/ffb30ded7afa402d8e10f66b50a144ec => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:54.679997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7b55f-4cc1-40ca-a360-932f20e0e75a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:54.681660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bcc7b55f-4cc1-40ca-a360-932f20e0e75a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:54.691814 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:54.696167 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7b55f-4cc1-40ca-a360-932f20e0e75a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:54.767707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741da37-935e-410b-9ecd-8c20923c0f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:54.768584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741da37-935e-410b-9ecd-8c20923c0f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:54.768861 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741da37-935e-410b-9ecd-8c20923c0f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:54.769129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741da37-935e-410b-9ecd-8c20923c0f7e admin admin] PATH_INFO: `/v3/users/f49df70c726340f784dc7d75bb141cf4` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:54.770367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741da37-935e-410b-9ecd-8c20923c0f7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f49df70c726340f784dc7d75bb141cf4)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:54.774902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741da37-935e-410b-9ecd-8c20923c0f7e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:54.831116 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d741da37-935e-410b-9ecd-8c20923c0f7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f49df70c726340f784dc7d75bb141cf4'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:54.853897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d741da37-935e-410b-9ecd-8c20923c0f7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f49df70c726340f784dc7d75bb141cf4'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:54.882469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d741da37-935e-410b-9ecd-8c20923c0f7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f49df70c726340f784dc7d75bb141cf4'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:54.883379 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d741da37-935e-410b-9ecd-8c20923c0f7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f49df70c726340f784dc7d75bb141cf4'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:54.899718 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d741da37-935e-410b-9ecd-8c20923c0f7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f49df70c726340f784dc7d75bb141cf4'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:54.910036 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 884/1810] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:42:54 2022] DELETE /identity/v3/users/f49df70c726340f784dc7d75bb141cf4 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:55.750484 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:55.752255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:55.766086 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d28c49-319a-4fe2-be35-d0fcb1ccc6ea admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:55.820637 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:55.838055 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138ba7c-1525-4808-887b-84c8e8b86035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:55.852154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138ba7c-1525-4808-887b-84c8e8b86035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:55.852587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138ba7c-1525-4808-887b-84c8e8b86035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:55.852973 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138ba7c-1525-4808-887b-84c8e8b86035 admin admin] PATH_INFO: `/v3/projects/f2233b85afde4af690fb8f474c4e459f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:55.855105 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138ba7c-1525-4808-887b-84c8e8b86035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2233b85afde4af690fb8f474c4e459f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:55.858116 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138ba7c-1525-4808-887b-84c8e8b86035 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:55.921731 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7138ba7c-1525-4808-887b-84c8e8b86035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2233b85afde4af690fb8f474c4e459f', 'request_id': 'req-7138ba7c-1525-4808-887b-84c8e8b86035'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:55.933315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7138ba7c-1525-4808-887b-84c8e8b86035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2233b85afde4af690fb8f474c4e459f', 'request_id': 'req-7138ba7c-1525-4808-887b-84c8e8b86035'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:55.942804 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7138ba7c-1525-4808-887b-84c8e8b86035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2233b85afde4af690fb8f474c4e459f', 'request_id': 'req-7138ba7c-1525-4808-887b-84c8e8b86035'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:55.957681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7138ba7c-1525-4808-887b-84c8e8b86035 admin admin] The token cache is being invalidate because project f2233b85afde4af690fb8f474c4e459f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:42:55.958103 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7138ba7c-1525-4808-887b-84c8e8b86035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:55.958869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7138ba7c-1525-4808-887b-84c8e8b86035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:55.960743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 927/1811] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:42:55 2022] DELETE /identity/v3/projects/f2233b85afde4af690fb8f474c4e459f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:55.967140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76033025-1daf-4e94-987e-17b709d9282d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:55.967563 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76033025-1daf-4e94-987e-17b709d9282d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:55.967636 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76033025-1daf-4e94-987e-17b709d9282d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:56.010262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-76033025-1daf-4e94-987e-17b709d9282d None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:56.012004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-76033025-1daf-4e94-987e-17b709d9282d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:56.110938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fc904-6dcc-48c5-bbf5-f642c2a68d56 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:56.111239 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fc904-6dcc-48c5-bbf5-f642c2a68d56 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:56.111511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fc904-6dcc-48c5-bbf5-f642c2a68d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:56.122130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 885/1812] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:56.145572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-188fc904-6dcc-48c5-bbf5-f642c2a68d56 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:56.146986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-188fc904-6dcc-48c5-bbf5-f642c2a68d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:56.159957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 928/1813] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:56.160838 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76033025-1daf-4e94-987e-17b709d9282d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:56.171441 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76033025-1daf-4e94-987e-17b709d9282d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:56.177756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea29e2-c546-4b31-96f9-2a4342b28547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:56.178638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea29e2-c546-4b31-96f9-2a4342b28547 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:56.178953 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea29e2-c546-4b31-96f9-2a4342b28547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:56.179298 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea29e2-c546-4b31-96f9-2a4342b28547 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:56.180109 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ea29e2-c546-4b31-96f9-2a4342b28547 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:42:56.180491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ea29e2-c546-4b31-96f9-2a4342b28547 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:56.184420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ea29e2-c546-4b31-96f9-2a4342b28547 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:56.205007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 886/1814] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:42:56.209758 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fc904-6dcc-48c5-bbf5-f642c2a68d56 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:56.219960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fc904-6dcc-48c5-bbf5-f642c2a68d56 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:56.226631 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b08cee-7d65-4ecd-992b-f27ff576fa17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:56.227311 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b08cee-7d65-4ecd-992b-f27ff576fa17 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:56.227600 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b08cee-7d65-4ecd-992b-f27ff576fa17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:56.227881 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b08cee-7d65-4ecd-992b-f27ff576fa17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:56.228764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b08cee-7d65-4ecd-992b-f27ff576fa17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:56.230736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b08cee-7d65-4ecd-992b-f27ff576fa17 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:56.285034 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 929/1815] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:42:56 2022] POST /identity/v3/projects => generated 389 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:56.289997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea29e2-c546-4b31-96f9-2a4342b28547 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:56.306680 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea29e2-c546-4b31-96f9-2a4342b28547 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:56.319376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:56.413533 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5970c1-c741-4391-ab1d-de755addddbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:56.415937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5970c1-c741-4391-ab1d-de755addddbd admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:56.416393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5970c1-c741-4391-ab1d-de755addddbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:56.417156 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5970c1-c741-4391-ab1d-de755addddbd admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:56.419788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5970c1-c741-4391-ab1d-de755addddbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:56.430593 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5970c1-c741-4391-ab1d-de755addddbd admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:56.495648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 887/1816] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:42:56 2022] POST /identity/v3/users => generated 295 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:42:56.497599 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b08cee-7d65-4ecd-992b-f27ff576fa17 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:56.515800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b08cee-7d65-4ecd-992b-f27ff576fa17 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:56.546518 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254f38a-be65-42ad-bfbe-ddf07a896896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:56.547400 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254f38a-be65-42ad-bfbe-ddf07a896896 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:56.547692 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254f38a-be65-42ad-bfbe-ddf07a896896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:56.548123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254f38a-be65-42ad-bfbe-ddf07a896896 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:56.548878 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254f38a-be65-42ad-bfbe-ddf07a896896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:56.551790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254f38a-be65-42ad-bfbe-ddf07a896896 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:56.571568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 930/1817] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:56.576102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5970c1-c741-4391-ab1d-de755addddbd admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:56.590906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5970c1-c741-4391-ab1d-de755addddbd admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:56.610457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4ead5-82b4-4d50-a0a5-aed366bc60cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:56.611938 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4ead5-82b4-4d50-a0a5-aed366bc60cb admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:56.612327 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4ead5-82b4-4d50-a0a5-aed366bc60cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:56.612617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4ead5-82b4-4d50-a0a5-aed366bc60cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:56.614364 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4ead5-82b4-4d50-a0a5-aed366bc60cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:56.616776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4ead5-82b4-4d50-a0a5-aed366bc60cb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:56.637999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 888/1818] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:56.642086 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254f38a-be65-42ad-bfbe-ddf07a896896 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:56.658419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254f38a-be65-42ad-bfbe-ddf07a896896 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:56.665362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a7a87-d35f-4766-bf59-a7225ca1ee38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:56.666549 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a7a87-d35f-4766-bf59-a7225ca1ee38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:56.667021 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a7a87-d35f-4766-bf59-a7225ca1ee38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:56.667449 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a7a87-d35f-4766-bf59-a7225ca1ee38 admin admin] PATH_INFO: `/v3/projects/a5121b3edb974becaa17c79c5e9bc941/users/4c4d7ed712d543eeaf1085a75dca4207/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:56.688496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a7a87-d35f-4766-bf59-a7225ca1ee38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5121b3edb974becaa17c79c5e9bc941, user_id=4c4d7ed712d543eeaf1085a75dca4207, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:56.691507 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a7a87-d35f-4766-bf59-a7225ca1ee38 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:56.717997 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 931/1819] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:42:56 2022] PUT /identity/v3/projects/a5121b3edb974becaa17c79c5e9bc941/users/4c4d7ed712d543eeaf1085a75dca4207/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:56.730856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecb793-dc86-4d34-929c-dc095d84ead9 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:56.731228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecb793-dc86-4d34-929c-dc095d84ead9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:56.731576 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecb793-dc86-4d34-929c-dc095d84ead9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:56.764232 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8cecb793-dc86-4d34-929c-dc095d84ead9 None None] MFA Rules not processed for user `4c4d7ed712d543eeaf1085a75dca4207`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:56.765621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8cecb793-dc86-4d34-929c-dc095d84ead9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:56.850183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 889/1820] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:56.855072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a7a87-d35f-4766-bf59-a7225ca1ee38 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:56.972052 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a7a87-d35f-4766-bf59-a7225ca1ee38 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:56.979489 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81aa7c-db40-4ccc-bf04-1b172e2eb085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:56.980443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81aa7c-db40-4ccc-bf04-1b172e2eb085 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:56.981160 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81aa7c-db40-4ccc-bf04-1b172e2eb085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:56.981748 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81aa7c-db40-4ccc-bf04-1b172e2eb085 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:56.982823 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81aa7c-db40-4ccc-bf04-1b172e2eb085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:56.985223 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81aa7c-db40-4ccc-bf04-1b172e2eb085 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:57.046206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 932/1821] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:42:56 2022] POST /identity/v3/projects => generated 389 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:57.050097 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecb793-dc86-4d34-929c-dc095d84ead9 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:57.063278 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecb793-dc86-4d34-929c-dc095d84ead9 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:57.128265 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87022c40-8bf3-4bd0-b605-f122980e4c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:57.129069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87022c40-8bf3-4bd0-b605-f122980e4c8b admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:57.129355 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87022c40-8bf3-4bd0-b605-f122980e4c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:57.129629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87022c40-8bf3-4bd0-b605-f122980e4c8b admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:57.136433 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87022c40-8bf3-4bd0-b605-f122980e4c8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:57.136433 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87022c40-8bf3-4bd0-b605-f122980e4c8b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:57.171735 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 890/1822] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:42:57 2022] POST /identity/v3/users => generated 301 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:57.172220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81aa7c-db40-4ccc-bf04-1b172e2eb085 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:57.180437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:57.188405 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81aa7c-db40-4ccc-bf04-1b172e2eb085 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:57.195666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dfd34-d6b3-46b5-ad87-165e27c74e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:57.196618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dfd34-d6b3-46b5-ad87-165e27c74e43 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:57.197247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dfd34-d6b3-46b5-ad87-165e27c74e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:57.197678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dfd34-d6b3-46b5-ad87-165e27c74e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:57.198630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17dfd34-d6b3-46b5-ad87-165e27c74e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:57.200700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17dfd34-d6b3-46b5-ad87-165e27c74e43 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:57.218136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 933/1823] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:57.219598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87022c40-8bf3-4bd0-b605-f122980e4c8b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:57.231804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87022c40-8bf3-4bd0-b605-f122980e4c8b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:57.237044 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28504a0d-f908-4c2f-8272-69b7a0fc5862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:57.237999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28504a0d-f908-4c2f-8272-69b7a0fc5862 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:57.238287 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28504a0d-f908-4c2f-8272-69b7a0fc5862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:57.238564 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28504a0d-f908-4c2f-8272-69b7a0fc5862 admin admin] PATH_INFO: `/v3/projects/c84ff8e2d299455eb95de7f508d17d3a/users/a1c787e801be4b2f953668616433baf0/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:57.244725 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:57.259949 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28504a0d-f908-4c2f-8272-69b7a0fc5862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84ff8e2d299455eb95de7f508d17d3a, user_id=a1c787e801be4b2f953668616433baf0, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:57.263398 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28504a0d-f908-4c2f-8272-69b7a0fc5862 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:57.280256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 891/1824] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:42:57 2022] PUT /identity/v3/projects/c84ff8e2d299455eb95de7f508d17d3a/users/a1c787e801be4b2f953668616433baf0/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:57.284429 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dfd34-d6b3-46b5-ad87-165e27c74e43 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:57.296366 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dfd34-d6b3-46b5-ad87-165e27c74e43 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:57.372547 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803baad-4b06-40cb-bf06-9ad52bdb1d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:57.373403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803baad-4b06-40cb-bf06-9ad52bdb1d05 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:57.373860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803baad-4b06-40cb-bf06-9ad52bdb1d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:57.374314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803baad-4b06-40cb-bf06-9ad52bdb1d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:57.375358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803baad-4b06-40cb-bf06-9ad52bdb1d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:57.377658 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803baad-4b06-40cb-bf06-9ad52bdb1d05 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:57.395526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28504a0d-f908-4c2f-8272-69b7a0fc5862 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:57.398109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 934/1825] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:57.412828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28504a0d-f908-4c2f-8272-69b7a0fc5862 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:57.419249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3c8af-a089-49ba-a564-4a1b92d51af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:57.420150 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3c8af-a089-49ba-a564-4a1b92d51af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:57.420432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3c8af-a089-49ba-a564-4a1b92d51af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:57.420711 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3c8af-a089-49ba-a564-4a1b92d51af3 admin admin] PATH_INFO: `/v3/projects/c84ff8e2d299455eb95de7f508d17d3a/users/a1c787e801be4b2f953668616433baf0/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:57.423750 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3c8af-a089-49ba-a564-4a1b92d51af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84ff8e2d299455eb95de7f508d17d3a, user_id=a1c787e801be4b2f953668616433baf0, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:57.428683 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3c8af-a089-49ba-a564-4a1b92d51af3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:57.444526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 892/1826] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:42:57 2022] PUT /identity/v3/projects/c84ff8e2d299455eb95de7f508d17d3a/users/a1c787e801be4b2f953668616433baf0/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:57.455850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e7ef2-7a16-4c4d-9768-1cb9d160f49f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:57.456143 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e7ef2-7a16-4c4d-9768-1cb9d160f49f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:57.456421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e7ef2-7a16-4c4d-9768-1cb9d160f49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:57.486208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d88e7ef2-7a16-4c4d-9768-1cb9d160f49f None None] MFA Rules not processed for user `a1c787e801be4b2f953668616433baf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:57.487363 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d88e7ef2-7a16-4c4d-9768-1cb9d160f49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:57.593552 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 935/1827] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:57.602892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3c8af-a089-49ba-a564-4a1b92d51af3 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:57.604609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-afb3c8af-a089-49ba-a564-4a1b92d51af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:57.686371 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3c8af-a089-49ba-a564-4a1b92d51af3 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:57.694191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43d72b-fb99-4f0b-aec8-5da1727c77e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:57.694191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43d72b-fb99-4f0b-aec8-5da1727c77e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:57.694191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43d72b-fb99-4f0b-aec8-5da1727c77e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:57.694191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43d72b-fb99-4f0b-aec8-5da1727c77e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:57.702385 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43d72b-fb99-4f0b-aec8-5da1727c77e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:57.703925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43d72b-fb99-4f0b-aec8-5da1727c77e0 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:57.715098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 893/1828] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:57.847473 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e7ef2-7a16-4c4d-9768-1cb9d160f49f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:57.923240 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e7ef2-7a16-4c4d-9768-1cb9d160f49f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:57.929940 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a1943-a730-43b3-ad77-387d1ff9565e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:57.930960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a1943-a730-43b3-ad77-387d1ff9565e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:57.931315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a1943-a730-43b3-ad77-387d1ff9565e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:57.931685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a1943-a730-43b3-ad77-387d1ff9565e admin admin] PATH_INFO: `/v3/users/4c4d7ed712d543eeaf1085a75dca4207` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:57.932999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a1943-a730-43b3-ad77-387d1ff9565e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c4d7ed712d543eeaf1085a75dca4207)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:57.935835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a1943-a730-43b3-ad77-387d1ff9565e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:57.954893 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:57.990241 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6d4a1943-a730-43b3-ad77-387d1ff9565e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c4d7ed712d543eeaf1085a75dca4207'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:57.990864 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6d4a1943-a730-43b3-ad77-387d1ff9565e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c4d7ed712d543eeaf1085a75dca4207'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:57.992217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6d4a1943-a730-43b3-ad77-387d1ff9565e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c4d7ed712d543eeaf1085a75dca4207'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:58.007241 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6d4a1943-a730-43b3-ad77-387d1ff9565e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c4d7ed712d543eeaf1085a75dca4207'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:58.014358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6d4a1943-a730-43b3-ad77-387d1ff9565e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c4d7ed712d543eeaf1085a75dca4207'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:58.028895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 936/1829] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:42:57 2022] DELETE /identity/v3/users/4c4d7ed712d543eeaf1085a75dca4207 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:58.034359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f5b77-9ac3-42fb-a92e-145b15cf68b8 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:58.034670 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f5b77-9ac3-42fb-a92e-145b15cf68b8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:58.034952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f5b77-9ac3-42fb-a92e-145b15cf68b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:58.058280 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-2f7f5b77-9ac3-42fb-a92e-145b15cf68b8 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:58.059661 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2f7f5b77-9ac3-42fb-a92e-145b15cf68b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:58.131816 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 894/1830] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:42:58.141999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a1943-a730-43b3-ad77-387d1ff9565e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:58.143619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6d4a1943-a730-43b3-ad77-387d1ff9565e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:58.204378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a1943-a730-43b3-ad77-387d1ff9565e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:58.209483 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308b408-af4b-41bf-89a7-1a4aaaafe0f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:58.210203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308b408-af4b-41bf-89a7-1a4aaaafe0f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:58.210489 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308b408-af4b-41bf-89a7-1a4aaaafe0f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:58.210764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308b408-af4b-41bf-89a7-1a4aaaafe0f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:58.230072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308b408-af4b-41bf-89a7-1a4aaaafe0f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:58.232110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308b408-af4b-41bf-89a7-1a4aaaafe0f2 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:58.262511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 937/1831] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:42:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:42:58.564330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f5b77-9ac3-42fb-a92e-145b15cf68b8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:58.565997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2f7f5b77-9ac3-42fb-a92e-145b15cf68b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:58.598068 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f5b77-9ac3-42fb-a92e-145b15cf68b8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:58.605768 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74844ac6-70b2-4780-89c3-fbe412a1ed8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:58.606701 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74844ac6-70b2-4780-89c3-fbe412a1ed8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:58.606970 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74844ac6-70b2-4780-89c3-fbe412a1ed8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:58.607306 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74844ac6-70b2-4780-89c3-fbe412a1ed8a admin admin] PATH_INFO: `/v3/projects/a5121b3edb974becaa17c79c5e9bc941` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:58.608459 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74844ac6-70b2-4780-89c3-fbe412a1ed8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5121b3edb974becaa17c79c5e9bc941)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:58.610443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74844ac6-70b2-4780-89c3-fbe412a1ed8a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:58.701235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-74844ac6-70b2-4780-89c3-fbe412a1ed8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5121b3edb974becaa17c79c5e9bc941', 'request_id': 'req-74844ac6-70b2-4780-89c3-fbe412a1ed8a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:58.707809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-74844ac6-70b2-4780-89c3-fbe412a1ed8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5121b3edb974becaa17c79c5e9bc941', 'request_id': 'req-74844ac6-70b2-4780-89c3-fbe412a1ed8a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:58.723649 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-74844ac6-70b2-4780-89c3-fbe412a1ed8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5121b3edb974becaa17c79c5e9bc941', 'request_id': 'req-74844ac6-70b2-4780-89c3-fbe412a1ed8a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:58.739052 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:58.749346 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-74844ac6-70b2-4780-89c3-fbe412a1ed8a admin admin] The token cache is being invalidate because project a5121b3edb974becaa17c79c5e9bc941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:42:58.751067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-74844ac6-70b2-4780-89c3-fbe412a1ed8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:58.753489 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-74844ac6-70b2-4780-89c3-fbe412a1ed8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:58.761534 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 895/1832] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:42:58 2022] DELETE /identity/v3/projects/a5121b3edb974becaa17c79c5e9bc941 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:58.762047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308b408-af4b-41bf-89a7-1a4aaaafe0f2 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:58.780213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308b408-af4b-41bf-89a7-1a4aaaafe0f2 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:58.916910 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df01c5-5116-42e8-9137-5b014317b0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:58.917813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df01c5-5116-42e8-9137-5b014317b0cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:58.918166 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df01c5-5116-42e8-9137-5b014317b0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:58.918447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df01c5-5116-42e8-9137-5b014317b0cb admin admin] PATH_INFO: `/v3/users/a1c787e801be4b2f953668616433baf0` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:58.919748 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df01c5-5116-42e8-9137-5b014317b0cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1c787e801be4b2f953668616433baf0)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:58.921741 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df01c5-5116-42e8-9137-5b014317b0cb admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:58.966190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:42:58.979751 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-19df01c5-5116-42e8-9137-5b014317b0cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1c787e801be4b2f953668616433baf0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:58.981471 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-19df01c5-5116-42e8-9137-5b014317b0cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1c787e801be4b2f953668616433baf0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:58.982138 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-19df01c5-5116-42e8-9137-5b014317b0cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1c787e801be4b2f953668616433baf0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:58.996733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-19df01c5-5116-42e8-9137-5b014317b0cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1c787e801be4b2f953668616433baf0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:59.004356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-19df01c5-5116-42e8-9137-5b014317b0cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1c787e801be4b2f953668616433baf0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:59.035559 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 938/1833] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:42:58 2022] DELETE /identity/v3/users/a1c787e801be4b2f953668616433baf0 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:59.331536 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74844ac6-70b2-4780-89c3-fbe412a1ed8a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:59.333010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-74844ac6-70b2-4780-89c3-fbe412a1ed8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:59.350868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74844ac6-70b2-4780-89c3-fbe412a1ed8a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:59.408124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc81fc-59a2-4761-a0fb-6b0b043ff065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:59.408996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc81fc-59a2-4761-a0fb-6b0b043ff065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:59.409282 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc81fc-59a2-4761-a0fb-6b0b043ff065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:59.409562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc81fc-59a2-4761-a0fb-6b0b043ff065 admin admin] PATH_INFO: `/v3/projects/c84ff8e2d299455eb95de7f508d17d3a` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:59.410787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc81fc-59a2-4761-a0fb-6b0b043ff065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c84ff8e2d299455eb95de7f508d17d3a)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:59.413417 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc81fc-59a2-4761-a0fb-6b0b043ff065 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:59.459950 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-eebc81fc-59a2-4761-a0fb-6b0b043ff065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c84ff8e2d299455eb95de7f508d17d3a', 'request_id': 'req-eebc81fc-59a2-4761-a0fb-6b0b043ff065'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:59.465590 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-eebc81fc-59a2-4761-a0fb-6b0b043ff065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c84ff8e2d299455eb95de7f508d17d3a', 'request_id': 'req-eebc81fc-59a2-4761-a0fb-6b0b043ff065'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:59.473588 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-eebc81fc-59a2-4761-a0fb-6b0b043ff065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c84ff8e2d299455eb95de7f508d17d3a', 'request_id': 'req-eebc81fc-59a2-4761-a0fb-6b0b043ff065'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:59.485467 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-eebc81fc-59a2-4761-a0fb-6b0b043ff065 admin admin] The token cache is being invalidate because project c84ff8e2d299455eb95de7f508d17d3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:42:59.485741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-eebc81fc-59a2-4761-a0fb-6b0b043ff065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:59.486421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-eebc81fc-59a2-4761-a0fb-6b0b043ff065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:42:59.487985 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 896/1834] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:42:59 2022] DELETE /identity/v3/projects/c84ff8e2d299455eb95de7f508d17d3a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:59.493034 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2db743-1378-4567-b46a-7064a25ce102 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:59.493330 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2db743-1378-4567-b46a-7064a25ce102 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:59.493611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2db743-1378-4567-b46a-7064a25ce102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:59.511832 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3c2db743-1378-4567-b46a-7064a25ce102 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:59.513046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3c2db743-1378-4567-b46a-7064a25ce102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:59.575094 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 939/1835] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:42:59.583165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c7ed3-d0c2-4422-af6e-b55051687691 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:59.584020 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c7ed3-d0c2-4422-af6e-b55051687691 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:59.584301 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c7ed3-d0c2-4422-af6e-b55051687691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:59.604359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-901c7ed3-d0c2-4422-af6e-b55051687691 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:59.605657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-901c7ed3-d0c2-4422-af6e-b55051687691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:42:59.614492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 897/1836] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:42:59.618965 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2db743-1378-4567-b46a-7064a25ce102 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:59.629174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2db743-1378-4567-b46a-7064a25ce102 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:59.634367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b85e5d-8904-459c-af44-4bfb61307b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:59.635029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b85e5d-8904-459c-af44-4bfb61307b8b admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:59.635355 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b85e5d-8904-459c-af44-4bfb61307b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:59.635641 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b85e5d-8904-459c-af44-4bfb61307b8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:59.636367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b85e5d-8904-459c-af44-4bfb61307b8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:42:59.636686 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b85e5d-8904-459c-af44-4bfb61307b8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:59.638328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b85e5d-8904-459c-af44-4bfb61307b8b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:59.650908 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 940/1837] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:42:59.655320 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c7ed3-d0c2-4422-af6e-b55051687691 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:59.664903 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c7ed3-d0c2-4422-af6e-b55051687691 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:59.669524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60782c7e-a583-4589-b9c9-b1e05d261b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:59.670228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60782c7e-a583-4589-b9c9-b1e05d261b66 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:59.670528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60782c7e-a583-4589-b9c9-b1e05d261b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:59.670804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60782c7e-a583-4589-b9c9-b1e05d261b66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:59.671531 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60782c7e-a583-4589-b9c9-b1e05d261b66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:59.673483 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60782c7e-a583-4589-b9c9-b1e05d261b66 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:59.700599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 898/1838] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:42:59 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:59.704911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b85e5d-8904-459c-af44-4bfb61307b8b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:59.716173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b85e5d-8904-459c-af44-4bfb61307b8b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:59.771710 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c727bf-6eb9-468d-9fc4-5951feb87e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:59.772385 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c727bf-6eb9-468d-9fc4-5951feb87e44 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:59.772671 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c727bf-6eb9-468d-9fc4-5951feb87e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:59.772945 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c727bf-6eb9-468d-9fc4-5951feb87e44 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:59.773641 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c727bf-6eb9-468d-9fc4-5951feb87e44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:59.775617 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c727bf-6eb9-468d-9fc4-5951feb87e44 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:59.795393 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 941/1839] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:42:59 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:42:59.799582 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60782c7e-a583-4589-b9c9-b1e05d261b66 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:59.810614 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60782c7e-a583-4589-b9c9-b1e05d261b66 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:59.816122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7bb62-d310-490c-851a-7913c643cc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:59.816801 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7bb62-d310-490c-851a-7913c643cc66 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:59.817086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7bb62-d310-490c-851a-7913c643cc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:59.817387 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7bb62-d310-490c-851a-7913c643cc66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:59.818672 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7bb62-d310-490c-851a-7913c643cc66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:59.820279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7bb62-d310-490c-851a-7913c643cc66 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:59.835556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 899/1840] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:59.836614 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c727bf-6eb9-468d-9fc4-5951feb87e44 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:59.858563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c727bf-6eb9-468d-9fc4-5951feb87e44 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:59.865751 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab047e-1b84-4503-baa7-f65858a856f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:59.866747 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab047e-1b84-4503-baa7-f65858a856f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:59.867141 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab047e-1b84-4503-baa7-f65858a856f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:59.867527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab047e-1b84-4503-baa7-f65858a856f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:59.868397 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab047e-1b84-4503-baa7-f65858a856f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:59.870131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab047e-1b84-4503-baa7-f65858a856f7 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:59.881813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 942/1841] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:42:59.886025 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7bb62-d310-490c-851a-7913c643cc66 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:42:59.897240 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7bb62-d310-490c-851a-7913c643cc66 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:42:59.902213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db940ef-e34a-4934-a9e7-dddd0ac58a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:42:59.903134 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db940ef-e34a-4934-a9e7-dddd0ac58a8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:59.903420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db940ef-e34a-4934-a9e7-dddd0ac58a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:59.903697 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db940ef-e34a-4934-a9e7-dddd0ac58a8a admin admin] PATH_INFO: `/v3/projects/f2d3a6aedb1d407a83f925858b52b407/users/f08e8e49eae546418261e004da64288f/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:59.920569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db940ef-e34a-4934-a9e7-dddd0ac58a8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2d3a6aedb1d407a83f925858b52b407, user_id=f08e8e49eae546418261e004da64288f, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:42:59.923783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db940ef-e34a-4934-a9e7-dddd0ac58a8a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:42:59.941211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 900/1842] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:42:59 2022] PUT /identity/v3/projects/f2d3a6aedb1d407a83f925858b52b407/users/f08e8e49eae546418261e004da64288f/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:42:59.958919 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ae36e-7ee9-4443-b5a2-66c11c3d0003 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:42:59.959457 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ae36e-7ee9-4443-b5a2-66c11c3d0003 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:42:59.959918 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ae36e-7ee9-4443-b5a2-66c11c3d0003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:42:59.992269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-026ae36e-7ee9-4443-b5a2-66c11c3d0003 None None] MFA Rules not processed for user `f08e8e49eae546418261e004da64288f`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:42:59.993475 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-026ae36e-7ee9-4443-b5a2-66c11c3d0003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:00.004727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:00.087478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 943/1843] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:42:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:00.092138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db940ef-e34a-4934-a9e7-dddd0ac58a8a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:00.132630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:00.170151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db940ef-e34a-4934-a9e7-dddd0ac58a8a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:00.174846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e7fa6-b654-4842-9ad0-f1513d194cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:00.175548 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e7fa6-b654-4842-9ad0-f1513d194cad admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:00.175826 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e7fa6-b654-4842-9ad0-f1513d194cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:00.176098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e7fa6-b654-4842-9ad0-f1513d194cad admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:00.177304 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e7fa6-b654-4842-9ad0-f1513d194cad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:00.179284 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e7fa6-b654-4842-9ad0-f1513d194cad admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:00.210006 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 901/1844] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:00 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:00.219825 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ae36e-7ee9-4443-b5a2-66c11c3d0003 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:00.231388 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ae36e-7ee9-4443-b5a2-66c11c3d0003 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:00.319741 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e23dd8-68ad-45b1-b646-077152c2198a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:00.320453 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e23dd8-68ad-45b1-b646-077152c2198a admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:00.320755 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e23dd8-68ad-45b1-b646-077152c2198a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:00.321015 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e23dd8-68ad-45b1-b646-077152c2198a admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:00.321739 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e23dd8-68ad-45b1-b646-077152c2198a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:00.329602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e23dd8-68ad-45b1-b646-077152c2198a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:00.360229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 944/1845] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:00 2022] POST /identity/v3/users => generated 300 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:00.362121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e7fa6-b654-4842-9ad0-f1513d194cad admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:00.374821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e7fa6-b654-4842-9ad0-f1513d194cad admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:00.380903 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4fcb0-35fe-419f-bcae-f0f1de9040b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:00.381413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4fcb0-35fe-419f-bcae-f0f1de9040b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:00.381714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4fcb0-35fe-419f-bcae-f0f1de9040b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:00.382012 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4fcb0-35fe-419f-bcae-f0f1de9040b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:00.382775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4fcb0-35fe-419f-bcae-f0f1de9040b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:00.384431 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4fcb0-35fe-419f-bcae-f0f1de9040b0 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:00.398397 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 902/1846] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:43:00.403032 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e23dd8-68ad-45b1-b646-077152c2198a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:00.414538 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e23dd8-68ad-45b1-b646-077152c2198a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:00.419465 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec768d-83a8-479f-873c-2f2bdf6ea12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:00.420386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec768d-83a8-479f-873c-2f2bdf6ea12b admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:00.420674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec768d-83a8-479f-873c-2f2bdf6ea12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:00.420950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec768d-83a8-479f-873c-2f2bdf6ea12b admin admin] PATH_INFO: `/v3/projects/1d646b44e808456087a35e750304aa9e/users/e1952ca8430d4bb5a6b12731465410cd/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:00.437188 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec768d-83a8-479f-873c-2f2bdf6ea12b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d646b44e808456087a35e750304aa9e, user_id=e1952ca8430d4bb5a6b12731465410cd, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:00.439906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec768d-83a8-479f-873c-2f2bdf6ea12b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:00.453977 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 945/1847] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:00 2022] PUT /identity/v3/projects/1d646b44e808456087a35e750304aa9e/users/e1952ca8430d4bb5a6b12731465410cd/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:00.458005 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4fcb0-35fe-419f-bcae-f0f1de9040b0 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:00.468154 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4fcb0-35fe-419f-bcae-f0f1de9040b0 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:00.528041 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df64d8-9d79-41d7-8cc7-306e0e9bd402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:00.529259 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df64d8-9d79-41d7-8cc7-306e0e9bd402 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:00.529259 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df64d8-9d79-41d7-8cc7-306e0e9bd402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:00.529459 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df64d8-9d79-41d7-8cc7-306e0e9bd402 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:00.530171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df64d8-9d79-41d7-8cc7-306e0e9bd402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:00.531733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df64d8-9d79-41d7-8cc7-306e0e9bd402 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:00.546142 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 903/1848] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:43:00.547395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec768d-83a8-479f-873c-2f2bdf6ea12b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:00.558887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec768d-83a8-479f-873c-2f2bdf6ea12b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:00.563965 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11a2a4-21d1-442c-b768-d105b63f3317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:00.564879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11a2a4-21d1-442c-b768-d105b63f3317 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:00.565158 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11a2a4-21d1-442c-b768-d105b63f3317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:00.565430 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11a2a4-21d1-442c-b768-d105b63f3317 admin admin] PATH_INFO: `/v3/projects/1d646b44e808456087a35e750304aa9e/users/e1952ca8430d4bb5a6b12731465410cd/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:00.567495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11a2a4-21d1-442c-b768-d105b63f3317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d646b44e808456087a35e750304aa9e, user_id=e1952ca8430d4bb5a6b12731465410cd, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:00.570314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11a2a4-21d1-442c-b768-d105b63f3317 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:00.581795 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 946/1849] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:00 2022] PUT /identity/v3/projects/1d646b44e808456087a35e750304aa9e/users/e1952ca8430d4bb5a6b12731465410cd/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:00.592190 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5c922-70e9-47a3-b1d4-023a98222f2e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:00.592486 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5c922-70e9-47a3-b1d4-023a98222f2e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:00.592768 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5c922-70e9-47a3-b1d4-023a98222f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:00.622696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ddd5c922-70e9-47a3-b1d4-023a98222f2e None None] MFA Rules not processed for user `e1952ca8430d4bb5a6b12731465410cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:00.624442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ddd5c922-70e9-47a3-b1d4-023a98222f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:00.703377 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 904/1850] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:00.715476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11a2a4-21d1-442c-b768-d105b63f3317 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:00.717239 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4d11a2a4-21d1-442c-b768-d105b63f3317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:00.784867 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11a2a4-21d1-442c-b768-d105b63f3317 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:00.789323 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd396e-349e-44fb-b906-74d1c0b34754 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:00.790018 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd396e-349e-44fb-b906-74d1c0b34754 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:00.790310 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd396e-349e-44fb-b906-74d1c0b34754 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:00.790587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd396e-349e-44fb-b906-74d1c0b34754 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:00.800331 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd396e-349e-44fb-b906-74d1c0b34754 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:00.801923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd396e-349e-44fb-b906-74d1c0b34754 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:00.811798 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 947/1851] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:01.470802 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5c922-70e9-47a3-b1d4-023a98222f2e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:01.492126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:01.536923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5c922-70e9-47a3-b1d4-023a98222f2e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:01.541448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e94a99-8a60-443f-a93d-431edf03ca2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:01.542357 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e94a99-8a60-443f-a93d-431edf03ca2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:01.542634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e94a99-8a60-443f-a93d-431edf03ca2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:01.542908 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e94a99-8a60-443f-a93d-431edf03ca2e admin admin] PATH_INFO: `/v3/users/f08e8e49eae546418261e004da64288f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:01.544091 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e94a99-8a60-443f-a93d-431edf03ca2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f08e8e49eae546418261e004da64288f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:01.546055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e94a99-8a60-443f-a93d-431edf03ca2e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:01.594496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-36e94a99-8a60-443f-a93d-431edf03ca2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f08e8e49eae546418261e004da64288f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:01.606977 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-36e94a99-8a60-443f-a93d-431edf03ca2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f08e8e49eae546418261e004da64288f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:01.614825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-36e94a99-8a60-443f-a93d-431edf03ca2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f08e8e49eae546418261e004da64288f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:01.615533 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-36e94a99-8a60-443f-a93d-431edf03ca2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f08e8e49eae546418261e004da64288f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:01.622878 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-36e94a99-8a60-443f-a93d-431edf03ca2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f08e8e49eae546418261e004da64288f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:01.629313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 905/1852] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:01 2022] DELETE /identity/v3/users/f08e8e49eae546418261e004da64288f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:01.633984 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1adfa3-00e9-40f6-8fec-e6f3e8afb885 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:01.634472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1adfa3-00e9-40f6-8fec-e6f3e8afb885 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:01.634572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1adfa3-00e9-40f6-8fec-e6f3e8afb885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:01.641188 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:01.655964 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-cd1adfa3-00e9-40f6-8fec-e6f3e8afb885 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:01.657436 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cd1adfa3-00e9-40f6-8fec-e6f3e8afb885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:01.718213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 948/1853] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:01.728688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e94a99-8a60-443f-a93d-431edf03ca2e admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:01.730179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-36e94a99-8a60-443f-a93d-431edf03ca2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:01.797130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e94a99-8a60-443f-a93d-431edf03ca2e admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:01.800904 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0b051-70c7-4a78-aca9-dc0358dcf1a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:01.801561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0b051-70c7-4a78-aca9-dc0358dcf1a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:01.801889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0b051-70c7-4a78-aca9-dc0358dcf1a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:01.802201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0b051-70c7-4a78-aca9-dc0358dcf1a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:01.811902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe0b051-70c7-4a78-aca9-dc0358dcf1a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:01.813349 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe0b051-70c7-4a78-aca9-dc0358dcf1a9 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:01.824167 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 906/1854] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:02.131579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1adfa3-00e9-40f6-8fec-e6f3e8afb885 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:02.133170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cd1adfa3-00e9-40f6-8fec-e6f3e8afb885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:02.152189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1adfa3-00e9-40f6-8fec-e6f3e8afb885 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:02.157421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8293a-4ca4-4408-aafb-3168370ee654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:02.158419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8293a-4ca4-4408-aafb-3168370ee654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:02.158694 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8293a-4ca4-4408-aafb-3168370ee654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:02.159066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8293a-4ca4-4408-aafb-3168370ee654 admin admin] PATH_INFO: `/v3/projects/f2d3a6aedb1d407a83f925858b52b407` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:02.160272 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8293a-4ca4-4408-aafb-3168370ee654 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2d3a6aedb1d407a83f925858b52b407)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:02.162486 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8293a-4ca4-4408-aafb-3168370ee654 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:02.211910 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f0c8293a-4ca4-4408-aafb-3168370ee654 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2d3a6aedb1d407a83f925858b52b407', 'request_id': 'req-f0c8293a-4ca4-4408-aafb-3168370ee654'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:02.220300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f0c8293a-4ca4-4408-aafb-3168370ee654 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2d3a6aedb1d407a83f925858b52b407', 'request_id': 'req-f0c8293a-4ca4-4408-aafb-3168370ee654'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:02.227214 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f0c8293a-4ca4-4408-aafb-3168370ee654 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2d3a6aedb1d407a83f925858b52b407', 'request_id': 'req-f0c8293a-4ca4-4408-aafb-3168370ee654'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:02.242844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f0c8293a-4ca4-4408-aafb-3168370ee654 admin admin] The token cache is being invalidate because project f2d3a6aedb1d407a83f925858b52b407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:02.243140 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f0c8293a-4ca4-4408-aafb-3168370ee654 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:02.243838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f0c8293a-4ca4-4408-aafb-3168370ee654 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:02.245487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 949/1855] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:02 2022] DELETE /identity/v3/projects/f2d3a6aedb1d407a83f925858b52b407 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:02.250868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0b051-70c7-4a78-aca9-dc0358dcf1a9 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:02.263516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0b051-70c7-4a78-aca9-dc0358dcf1a9 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:02.306200 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:02.329837 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d7957-354d-4980-a0d1-f68f4ec4c511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:02.330761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d7957-354d-4980-a0d1-f68f4ec4c511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:02.331178 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d7957-354d-4980-a0d1-f68f4ec4c511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:02.331461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d7957-354d-4980-a0d1-f68f4ec4c511 admin admin] PATH_INFO: `/v3/users/e1952ca8430d4bb5a6b12731465410cd` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:02.332688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d7957-354d-4980-a0d1-f68f4ec4c511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1952ca8430d4bb5a6b12731465410cd)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:02.334754 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d7957-354d-4980-a0d1-f68f4ec4c511 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:02.390952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a51d7957-354d-4980-a0d1-f68f4ec4c511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1952ca8430d4bb5a6b12731465410cd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:02.406674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a51d7957-354d-4980-a0d1-f68f4ec4c511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1952ca8430d4bb5a6b12731465410cd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:02.418905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a51d7957-354d-4980-a0d1-f68f4ec4c511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1952ca8430d4bb5a6b12731465410cd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:02.419805 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a51d7957-354d-4980-a0d1-f68f4ec4c511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1952ca8430d4bb5a6b12731465410cd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:02.432736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a51d7957-354d-4980-a0d1-f68f4ec4c511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1952ca8430d4bb5a6b12731465410cd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:02.441565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 907/1856] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:02 2022] DELETE /identity/v3/users/e1952ca8430d4bb5a6b12731465410cd => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:02.833335 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8293a-4ca4-4408-aafb-3168370ee654 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:02.834940 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f0c8293a-4ca4-4408-aafb-3168370ee654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:02.864933 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8293a-4ca4-4408-aafb-3168370ee654 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:02.925804 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc0523-434e-4509-a001-b88335638754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:02.926643 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc0523-434e-4509-a001-b88335638754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:02.927458 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc0523-434e-4509-a001-b88335638754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:02.928080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc0523-434e-4509-a001-b88335638754 admin admin] PATH_INFO: `/v3/projects/1d646b44e808456087a35e750304aa9e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:02.930168 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc0523-434e-4509-a001-b88335638754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d646b44e808456087a35e750304aa9e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:02.932102 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc0523-434e-4509-a001-b88335638754 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:02.970282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:02.986639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-10dc0523-434e-4509-a001-b88335638754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d646b44e808456087a35e750304aa9e', 'request_id': 'req-10dc0523-434e-4509-a001-b88335638754'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:02.994327 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-10dc0523-434e-4509-a001-b88335638754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d646b44e808456087a35e750304aa9e', 'request_id': 'req-10dc0523-434e-4509-a001-b88335638754'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:03.001279 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-10dc0523-434e-4509-a001-b88335638754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d646b44e808456087a35e750304aa9e', 'request_id': 'req-10dc0523-434e-4509-a001-b88335638754'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:03.012192 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-10dc0523-434e-4509-a001-b88335638754 admin admin] The token cache is being invalidate because project 1d646b44e808456087a35e750304aa9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:03.012488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-10dc0523-434e-4509-a001-b88335638754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:03.013536 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-10dc0523-434e-4509-a001-b88335638754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:03.014749 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 950/1857] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:02 2022] DELETE /identity/v3/projects/1d646b44e808456087a35e750304aa9e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:03.020017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d81faa-4dd9-4ae1-bf68-19fbe40cd665 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:03.021000 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d81faa-4dd9-4ae1-bf68-19fbe40cd665 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:03.021084 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d81faa-4dd9-4ae1-bf68-19fbe40cd665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:03.040759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-26d81faa-4dd9-4ae1-bf68-19fbe40cd665 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:03.042007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-26d81faa-4dd9-4ae1-bf68-19fbe40cd665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:03.102724 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 908/1858] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:03.112235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516fda9-f5ba-44d2-8266-0998c254f8ad None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:03.118029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516fda9-f5ba-44d2-8266-0998c254f8ad None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:03.118029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516fda9-f5ba-44d2-8266-0998c254f8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:03.132351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-0516fda9-f5ba-44d2-8266-0998c254f8ad None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:03.133591 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0516fda9-f5ba-44d2-8266-0998c254f8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:03.141780 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 951/1859] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:03.146765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d81faa-4dd9-4ae1-bf68-19fbe40cd665 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:03.156990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d81faa-4dd9-4ae1-bf68-19fbe40cd665 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:03.165277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15a293-f055-4d23-a6e1-a3b80e7574aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:03.166120 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15a293-f055-4d23-a6e1-a3b80e7574aa admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:03.166504 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15a293-f055-4d23-a6e1-a3b80e7574aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:03.166897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15a293-f055-4d23-a6e1-a3b80e7574aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:03.167733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15a293-f055-4d23-a6e1-a3b80e7574aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:43:03.168142 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15a293-f055-4d23-a6e1-a3b80e7574aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:03.170319 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15a293-f055-4d23-a6e1-a3b80e7574aa admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:03.183431 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 909/1860] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:43:03.187894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516fda9-f5ba-44d2-8266-0998c254f8ad None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:03.197422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516fda9-f5ba-44d2-8266-0998c254f8ad None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:03.203992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c23206-6cbb-401e-b9b2-edaed626e6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:03.204668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c23206-6cbb-401e-b9b2-edaed626e6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:03.204957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c23206-6cbb-401e-b9b2-edaed626e6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:03.205238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c23206-6cbb-401e-b9b2-edaed626e6e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:03.205945 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c23206-6cbb-401e-b9b2-edaed626e6e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:03.208439 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c23206-6cbb-401e-b9b2-edaed626e6e5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:03.236203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 952/1861] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:03 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:03.240360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15a293-f055-4d23-a6e1-a3b80e7574aa admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:03.251307 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15a293-f055-4d23-a6e1-a3b80e7574aa admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:03.304299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:03.320761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019976e-89b1-4da0-8166-0803bfb3dcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:03.321619 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019976e-89b1-4da0-8166-0803bfb3dcf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:03.322082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019976e-89b1-4da0-8166-0803bfb3dcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:03.322463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019976e-89b1-4da0-8166-0803bfb3dcf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:03.323274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5019976e-89b1-4da0-8166-0803bfb3dcf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:03.325367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5019976e-89b1-4da0-8166-0803bfb3dcf6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:03.352513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 910/1862] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:03 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:43:03.359831 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c23206-6cbb-401e-b9b2-edaed626e6e5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:03.372303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c23206-6cbb-401e-b9b2-edaed626e6e5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:03.381139 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3be76b-0b07-43da-8fa8-076c6abb9a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:03.381849 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3be76b-0b07-43da-8fa8-076c6abb9a2b admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:03.382638 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3be76b-0b07-43da-8fa8-076c6abb9a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:03.382638 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3be76b-0b07-43da-8fa8-076c6abb9a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:03.384282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3be76b-0b07-43da-8fa8-076c6abb9a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:03.389713 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3be76b-0b07-43da-8fa8-076c6abb9a2b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:03.404052 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 953/1863] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:03.408110 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019976e-89b1-4da0-8166-0803bfb3dcf6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:03.423777 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019976e-89b1-4da0-8166-0803bfb3dcf6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:03.435803 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28800e64-ae2d-41e2-9707-d1e952656f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:03.436499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28800e64-ae2d-41e2-9707-d1e952656f29 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:03.436787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28800e64-ae2d-41e2-9707-d1e952656f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:03.437105 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28800e64-ae2d-41e2-9707-d1e952656f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:03.437807 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28800e64-ae2d-41e2-9707-d1e952656f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:03.439563 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28800e64-ae2d-41e2-9707-d1e952656f29 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:03.457267 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 911/1864] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:03.461952 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3be76b-0b07-43da-8fa8-076c6abb9a2b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:03.495259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3be76b-0b07-43da-8fa8-076c6abb9a2b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:03.501261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45795c77-6b22-411c-a117-6b804875b4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:03.502535 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45795c77-6b22-411c-a117-6b804875b4d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:03.502818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45795c77-6b22-411c-a117-6b804875b4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:03.503094 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45795c77-6b22-411c-a117-6b804875b4d9 admin admin] PATH_INFO: `/v3/projects/9c2c209b8ddf41bdbed5b71206691f76/users/eb0452030f2045d19db1336543365003/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:03.519298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45795c77-6b22-411c-a117-6b804875b4d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c2c209b8ddf41bdbed5b71206691f76, user_id=eb0452030f2045d19db1336543365003, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:03.522974 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45795c77-6b22-411c-a117-6b804875b4d9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:03.541577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 954/1865] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:03 2022] PUT /identity/v3/projects/9c2c209b8ddf41bdbed5b71206691f76/users/eb0452030f2045d19db1336543365003/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:03.552673 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffaa0e-f30c-44a6-82f0-ebb4404fb2d1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:03.553093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffaa0e-f30c-44a6-82f0-ebb4404fb2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:03.553513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffaa0e-f30c-44a6-82f0-ebb4404fb2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:03.603814 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-97ffaa0e-f30c-44a6-82f0-ebb4404fb2d1 None None] MFA Rules not processed for user `eb0452030f2045d19db1336543365003`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:03.605630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-97ffaa0e-f30c-44a6-82f0-ebb4404fb2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:03.767157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 912/1866] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:03.780032 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45795c77-6b22-411c-a117-6b804875b4d9 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:03.782286 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-45795c77-6b22-411c-a117-6b804875b4d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:03.868781 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45795c77-6b22-411c-a117-6b804875b4d9 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:03.873254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ed2ad-a668-4b46-a585-0e096432d4ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:03.873914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ed2ad-a668-4b46-a585-0e096432d4ad service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:03.874233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ed2ad-a668-4b46-a585-0e096432d4ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:03.874745 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ed2ad-a668-4b46-a585-0e096432d4ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:03.884781 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ed2ad-a668-4b46-a585-0e096432d4ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:03.886282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ed2ad-a668-4b46-a585-0e096432d4ad service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:03.895855 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 955/1867] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:03.915901 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ffaa0e-f30c-44a6-82f0-ebb4404fb2d1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:03.984489 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ffaa0e-f30c-44a6-82f0-ebb4404fb2d1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:03.989139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7eb0a-e5d7-494e-ae0b-2ab0a50e62d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:03.990587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7eb0a-e5d7-494e-ae0b-2ab0a50e62d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:03.990872 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7eb0a-e5d7-494e-ae0b-2ab0a50e62d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:03.991168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7eb0a-e5d7-494e-ae0b-2ab0a50e62d6 admin admin] PATH_INFO: `/v3/users/eb0452030f2045d19db1336543365003` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:03.992372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7eb0a-e5d7-494e-ae0b-2ab0a50e62d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb0452030f2045d19db1336543365003)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:03.994355 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7eb0a-e5d7-494e-ae0b-2ab0a50e62d6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:04.041920 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a8c7eb0a-e5d7-494e-ae0b-2ab0a50e62d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb0452030f2045d19db1336543365003'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:04.049426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:04.054868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a8c7eb0a-e5d7-494e-ae0b-2ab0a50e62d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb0452030f2045d19db1336543365003'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:04.062662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a8c7eb0a-e5d7-494e-ae0b-2ab0a50e62d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb0452030f2045d19db1336543365003'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:04.063388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a8c7eb0a-e5d7-494e-ae0b-2ab0a50e62d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb0452030f2045d19db1336543365003'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:04.070818 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a8c7eb0a-e5d7-494e-ae0b-2ab0a50e62d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb0452030f2045d19db1336543365003'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:04.077551 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 913/1868] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:03 2022] DELETE /identity/v3/users/eb0452030f2045d19db1336543365003 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:04.082240 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f5dd0-b8d5-43cd-88df-fc3b43a62b76 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:04.082540 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f5dd0-b8d5-43cd-88df-fc3b43a62b76 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:04.082827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f5dd0-b8d5-43cd-88df-fc3b43a62b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:04.103002 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-305f5dd0-b8d5-43cd-88df-fc3b43a62b76 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:04.104218 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-305f5dd0-b8d5-43cd-88df-fc3b43a62b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:04.157884 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:04.169626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 956/1869] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:04.179942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7eb0a-e5d7-494e-ae0b-2ab0a50e62d6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:04.181361 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a8c7eb0a-e5d7-494e-ae0b-2ab0a50e62d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:04.241502 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7eb0a-e5d7-494e-ae0b-2ab0a50e62d6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:04.246264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9251a-1561-4cf5-b237-1670022d9525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:04.247032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9251a-1561-4cf5-b237-1670022d9525 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:04.247394 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9251a-1561-4cf5-b237-1670022d9525 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:04.247750 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9251a-1561-4cf5-b237-1670022d9525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:04.258045 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f9251a-1561-4cf5-b237-1670022d9525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:04.259488 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f9251a-1561-4cf5-b237-1670022d9525 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:04.269289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 914/1870] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:04.552033 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f5dd0-b8d5-43cd-88df-fc3b43a62b76 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:04.553753 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-305f5dd0-b8d5-43cd-88df-fc3b43a62b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:04.579810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f5dd0-b8d5-43cd-88df-fc3b43a62b76 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:04.585704 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419d2f6-bcbe-431e-96e3-739fb5c5c3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:04.586613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419d2f6-bcbe-431e-96e3-739fb5c5c3b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:04.586903 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419d2f6-bcbe-431e-96e3-739fb5c5c3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:04.587183 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419d2f6-bcbe-431e-96e3-739fb5c5c3b0 admin admin] PATH_INFO: `/v3/projects/9c2c209b8ddf41bdbed5b71206691f76` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:04.588405 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419d2f6-bcbe-431e-96e3-739fb5c5c3b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c2c209b8ddf41bdbed5b71206691f76)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:04.590372 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419d2f6-bcbe-431e-96e3-739fb5c5c3b0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:04.644703 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7419d2f6-bcbe-431e-96e3-739fb5c5c3b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c2c209b8ddf41bdbed5b71206691f76', 'request_id': 'req-7419d2f6-bcbe-431e-96e3-739fb5c5c3b0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:04.655101 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7419d2f6-bcbe-431e-96e3-739fb5c5c3b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c2c209b8ddf41bdbed5b71206691f76', 'request_id': 'req-7419d2f6-bcbe-431e-96e3-739fb5c5c3b0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:04.663132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7419d2f6-bcbe-431e-96e3-739fb5c5c3b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c2c209b8ddf41bdbed5b71206691f76', 'request_id': 'req-7419d2f6-bcbe-431e-96e3-739fb5c5c3b0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:04.677371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7419d2f6-bcbe-431e-96e3-739fb5c5c3b0 admin admin] The token cache is being invalidate because project 9c2c209b8ddf41bdbed5b71206691f76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:04.677655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7419d2f6-bcbe-431e-96e3-739fb5c5c3b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:04.678359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7419d2f6-bcbe-431e-96e3-739fb5c5c3b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:04.678987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9251a-1561-4cf5-b237-1670022d9525 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:04.683215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 957/1871] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:04 2022] DELETE /identity/v3/projects/9c2c209b8ddf41bdbed5b71206691f76 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:04.683327 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c7f9251a-1561-4cf5-b237-1670022d9525 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:04.684573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbdbc0-231a-4378-b6ac-e83ce7f5b77f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:04.684864 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbdbc0-231a-4378-b6ac-e83ce7f5b77f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:04.685306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbdbc0-231a-4378-b6ac-e83ce7f5b77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:04.697182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9251a-1561-4cf5-b237-1670022d9525 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:04.704654 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-18bbdbc0-231a-4378-b6ac-e83ce7f5b77f None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:04.706881 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-18bbdbc0-231a-4378-b6ac-e83ce7f5b77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:04.778710 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1eca3-68ae-42d0-8f02-7fee31f16e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:04.780030 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1eca3-68ae-42d0-8f02-7fee31f16e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:04.780510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1eca3-68ae-42d0-8f02-7fee31f16e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:04.780972 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1eca3-68ae-42d0-8f02-7fee31f16e18 admin admin] PATH_INFO: `/v3/users/000408e897c644c6bc13854132d2b5b3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:04.783013 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1eca3-68ae-42d0-8f02-7fee31f16e18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=000408e897c644c6bc13854132d2b5b3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:04.785460 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1eca3-68ae-42d0-8f02-7fee31f16e18 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:04.793811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 958/1872] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:04.801797 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036db66-2e38-4a23-9f9c-a9f837eb5ad3 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:04.802121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036db66-2e38-4a23-9f9c-a9f837eb5ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:04.803176 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036db66-2e38-4a23-9f9c-a9f837eb5ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:04.825528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c036db66-2e38-4a23-9f9c-a9f837eb5ad3 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:04.826998 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c036db66-2e38-4a23-9f9c-a9f837eb5ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:04.840350 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 959/1873] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:43:04.841278 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036db66-2e38-4a23-9f9c-a9f837eb5ad3 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:04.855767 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-33f1eca3-68ae-42d0-8f02-7fee31f16e18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '000408e897c644c6bc13854132d2b5b3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:04.855975 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036db66-2e38-4a23-9f9c-a9f837eb5ad3 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:04.865042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f518d-0e53-4ffa-8f8f-8b20085b2b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:04.865715 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f518d-0e53-4ffa-8f8f-8b20085b2b5a admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:04.866033 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f518d-0e53-4ffa-8f8f-8b20085b2b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:04.866333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f518d-0e53-4ffa-8f8f-8b20085b2b5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:04.867059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f518d-0e53-4ffa-8f8f-8b20085b2b5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:43:04.867374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f518d-0e53-4ffa-8f8f-8b20085b2b5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:04.869212 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f518d-0e53-4ffa-8f8f-8b20085b2b5a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:04.873637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-33f1eca3-68ae-42d0-8f02-7fee31f16e18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '000408e897c644c6bc13854132d2b5b3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:04.883198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 960/1874] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:43:04.885984 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-33f1eca3-68ae-42d0-8f02-7fee31f16e18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '000408e897c644c6bc13854132d2b5b3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:04.887482 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-33f1eca3-68ae-42d0-8f02-7fee31f16e18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '000408e897c644c6bc13854132d2b5b3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:04.887619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f518d-0e53-4ffa-8f8f-8b20085b2b5a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:04.900669 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f518d-0e53-4ffa-8f8f-8b20085b2b5a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:04.902420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-33f1eca3-68ae-42d0-8f02-7fee31f16e18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '000408e897c644c6bc13854132d2b5b3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:04.904534 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:04.912554 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 915/1875] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:04 2022] DELETE /identity/v3/users/000408e897c644c6bc13854132d2b5b3 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:43:04.915828 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac01c7e-032d-4128-b06f-bd767eaaf7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:04.915828 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac01c7e-032d-4128-b06f-bd767eaaf7db admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:04.915828 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac01c7e-032d-4128-b06f-bd767eaaf7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:04.915828 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac01c7e-032d-4128-b06f-bd767eaaf7db admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:04.916208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac01c7e-032d-4128-b06f-bd767eaaf7db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:04.917557 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4eee2-dbc3-4672-8328-4373258148cb None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:04.918614 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4eee2-dbc3-4672-8328-4373258148cb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:04.919100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4eee2-dbc3-4672-8328-4373258148cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:04.921061 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac01c7e-032d-4128-b06f-bd767eaaf7db admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:04.945873 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0bd4eee2-dbc3-4672-8328-4373258148cb None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:04.947834 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0bd4eee2-dbc3-4672-8328-4373258148cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:04.963911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 961/1876] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:04 2022] POST /identity/v3/projects => generated 399 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:04.968592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac01c7e-032d-4128-b06f-bd767eaaf7db admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:04.971813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fac01c7e-032d-4128-b06f-bd767eaaf7db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:05.048768 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:05.049323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 916/1877] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:43:05.066356 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4eee2-dbc3-4672-8328-4373258148cb None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:05.068722 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0bd4eee2-dbc3-4672-8328-4373258148cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:05.100830 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac01c7e-032d-4128-b06f-bd767eaaf7db admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:05.106416 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e58a2-8f7e-4061-a115-1026f74ce23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:05.107221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e58a2-8f7e-4061-a115-1026f74ce23e admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:05.107605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e58a2-8f7e-4061-a115-1026f74ce23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:05.107893 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e58a2-8f7e-4061-a115-1026f74ce23e admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:05.108647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e58a2-8f7e-4061-a115-1026f74ce23e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:05.111089 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e58a2-8f7e-4061-a115-1026f74ce23e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:05.145221 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4eee2-dbc3-4672-8328-4373258148cb None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:05.152085 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 962/1878] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:04 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:05.165050 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e58a2-8f7e-4061-a115-1026f74ce23e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:05.177858 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e58a2-8f7e-4061-a115-1026f74ce23e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:05.184430 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724953a-f838-4871-a250-47fde655f340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:05.185921 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724953a-f838-4871-a250-47fde655f340 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:05.186708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724953a-f838-4871-a250-47fde655f340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:05.187368 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724953a-f838-4871-a250-47fde655f340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:05.188577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724953a-f838-4871-a250-47fde655f340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:05.192683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724953a-f838-4871-a250-47fde655f340 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:05.209746 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 963/1879] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:05.214282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724953a-f838-4871-a250-47fde655f340 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:05.227810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724953a-f838-4871-a250-47fde655f340 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:05.233078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb81ce-fb81-43d9-b8db-c7884e73c93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:05.233783 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb81ce-fb81-43d9-b8db-c7884e73c93e admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:05.234114 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb81ce-fb81-43d9-b8db-c7884e73c93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:05.234395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb81ce-fb81-43d9-b8db-c7884e73c93e admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:05.235127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb81ce-fb81-43d9-b8db-c7884e73c93e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:05.236745 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb81ce-fb81-43d9-b8db-c7884e73c93e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:05.249521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 964/1880] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:05.252342 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76833b-9fee-4d8a-a557-beab9209d2bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:05.253169 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76833b-9fee-4d8a-a557-beab9209d2bf service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:05.253472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76833b-9fee-4d8a-a557-beab9209d2bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:05.253890 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76833b-9fee-4d8a-a557-beab9209d2bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:05.260256 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb81ce-fb81-43d9-b8db-c7884e73c93e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:05.273896 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76833b-9fee-4d8a-a557-beab9209d2bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:05.275342 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb81ce-fb81-43d9-b8db-c7884e73c93e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:05.275872 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76833b-9fee-4d8a-a557-beab9209d2bf service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:05.280673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e27ee-85d1-425c-bb3e-9cf523aebf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:05.281679 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e27ee-85d1-425c-bb3e-9cf523aebf8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:05.282060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e27ee-85d1-425c-bb3e-9cf523aebf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:05.282582 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e27ee-85d1-425c-bb3e-9cf523aebf8f admin admin] PATH_INFO: `/v3/projects/64aa4006876542738b9594f1ffefc6fe/users/b5388e1730274b08bbeb8dbed7fad5d2/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:05.295592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 917/1881] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:43:05.304817 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e27ee-85d1-425c-bb3e-9cf523aebf8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=64aa4006876542738b9594f1ffefc6fe, user_id=b5388e1730274b08bbeb8dbed7fad5d2, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:05.307886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e27ee-85d1-425c-bb3e-9cf523aebf8f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:05.337778 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 965/1882] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:05 2022] PUT /identity/v3/projects/64aa4006876542738b9594f1ffefc6fe/users/b5388e1730274b08bbeb8dbed7fad5d2/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:05.350138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712bc0d3-a0d8-4801-908d-83d57a7feba5 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:05.350461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712bc0d3-a0d8-4801-908d-83d57a7feba5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:05.350726 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712bc0d3-a0d8-4801-908d-83d57a7feba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:05.383098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-712bc0d3-a0d8-4801-908d-83d57a7feba5 None None] MFA Rules not processed for user `b5388e1730274b08bbeb8dbed7fad5d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:05.384638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-712bc0d3-a0d8-4801-908d-83d57a7feba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:05.444910 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:05.473161 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 918/1883] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:05.485098 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e27ee-85d1-425c-bb3e-9cf523aebf8f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:05.487232 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ca7e27ee-85d1-425c-bb3e-9cf523aebf8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:05.566875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e27ee-85d1-425c-bb3e-9cf523aebf8f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:05.571268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5832fa-bc96-404a-8773-a8984d972193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:05.571934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5832fa-bc96-404a-8773-a8984d972193 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:05.572216 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5832fa-bc96-404a-8773-a8984d972193 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:05.572493 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5832fa-bc96-404a-8773-a8984d972193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:05.584833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5832fa-bc96-404a-8773-a8984d972193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:05.586677 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5832fa-bc96-404a-8773-a8984d972193 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:05.597665 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 966/1884] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:05 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:05.660523 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712bc0d3-a0d8-4801-908d-83d57a7feba5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:05.662295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-712bc0d3-a0d8-4801-908d-83d57a7feba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:05.771588 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712bc0d3-a0d8-4801-908d-83d57a7feba5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:05.776075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4a6f7-61cb-4460-8815-ff07097de220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:05.776959 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4a6f7-61cb-4460-8815-ff07097de220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:05.777277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4a6f7-61cb-4460-8815-ff07097de220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:05.777695 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4a6f7-61cb-4460-8815-ff07097de220 admin admin] PATH_INFO: `/v3/projects/c1efcc2c519f4699a67aefc2edad6593` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:05.779021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f4a6f7-61cb-4460-8815-ff07097de220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1efcc2c519f4699a67aefc2edad6593)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:05.781044 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f4a6f7-61cb-4460-8815-ff07097de220 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:05.856291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-99f4a6f7-61cb-4460-8815-ff07097de220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1efcc2c519f4699a67aefc2edad6593', 'request_id': 'req-99f4a6f7-61cb-4460-8815-ff07097de220'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:05.868042 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-99f4a6f7-61cb-4460-8815-ff07097de220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1efcc2c519f4699a67aefc2edad6593', 'request_id': 'req-99f4a6f7-61cb-4460-8815-ff07097de220'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:05.878245 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-99f4a6f7-61cb-4460-8815-ff07097de220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1efcc2c519f4699a67aefc2edad6593', 'request_id': 'req-99f4a6f7-61cb-4460-8815-ff07097de220'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:05.893435 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-99f4a6f7-61cb-4460-8815-ff07097de220 admin admin] The token cache is being invalidate because project c1efcc2c519f4699a67aefc2edad6593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:05.893732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-99f4a6f7-61cb-4460-8815-ff07097de220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:05.894736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-99f4a6f7-61cb-4460-8815-ff07097de220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:05.900700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 919/1885] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:05 2022] DELETE /identity/v3/projects/c1efcc2c519f4699a67aefc2edad6593 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:05.901586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5832fa-bc96-404a-8773-a8984d972193 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:05.917336 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5832fa-bc96-404a-8773-a8984d972193 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:06.036541 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd96631-d8a8-4f25-8791-310f56c627f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:06.038072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd96631-d8a8-4f25-8791-310f56c627f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:06.039755 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd96631-d8a8-4f25-8791-310f56c627f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:06.040101 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd96631-d8a8-4f25-8791-310f56c627f0 admin admin] PATH_INFO: `/v3/users/9d9f9fb81ae547f79563d0090188704c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:06.042789 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd96631-d8a8-4f25-8791-310f56c627f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d9f9fb81ae547f79563d0090188704c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:06.046741 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd96631-d8a8-4f25-8791-310f56c627f0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:06.130497 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:06.161674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4cd96631-d8a8-4f25-8791-310f56c627f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d9f9fb81ae547f79563d0090188704c'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:06.162362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4cd96631-d8a8-4f25-8791-310f56c627f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d9f9fb81ae547f79563d0090188704c'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:06.162989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4cd96631-d8a8-4f25-8791-310f56c627f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d9f9fb81ae547f79563d0090188704c'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:06.182207 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4cd96631-d8a8-4f25-8791-310f56c627f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d9f9fb81ae547f79563d0090188704c'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:06.202008 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4cd96631-d8a8-4f25-8791-310f56c627f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d9f9fb81ae547f79563d0090188704c'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:06.226019 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 967/1886] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:05 2022] DELETE /identity/v3/users/9d9f9fb81ae547f79563d0090188704c => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:07.059019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4a6f7-61cb-4460-8815-ff07097de220 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:07.061313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-99f4a6f7-61cb-4460-8815-ff07097de220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:07.076648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4a6f7-61cb-4460-8815-ff07097de220 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:07.143400 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958bacd-65d4-42d0-be1b-0551b5a2f33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:07.144275 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958bacd-65d4-42d0-be1b-0551b5a2f33a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:07.144556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958bacd-65d4-42d0-be1b-0551b5a2f33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:07.144829 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958bacd-65d4-42d0-be1b-0551b5a2f33a admin admin] PATH_INFO: `/v3/projects/1690e8e04a3b48ebbe99c45cbe8a28a2` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:07.146610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958bacd-65d4-42d0-be1b-0551b5a2f33a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1690e8e04a3b48ebbe99c45cbe8a28a2)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:07.148574 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958bacd-65d4-42d0-be1b-0551b5a2f33a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:07.173032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:07.210210 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e958bacd-65d4-42d0-be1b-0551b5a2f33a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1690e8e04a3b48ebbe99c45cbe8a28a2', 'request_id': 'req-e958bacd-65d4-42d0-be1b-0551b5a2f33a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:07.219439 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e958bacd-65d4-42d0-be1b-0551b5a2f33a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1690e8e04a3b48ebbe99c45cbe8a28a2', 'request_id': 'req-e958bacd-65d4-42d0-be1b-0551b5a2f33a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:07.237604 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e958bacd-65d4-42d0-be1b-0551b5a2f33a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1690e8e04a3b48ebbe99c45cbe8a28a2', 'request_id': 'req-e958bacd-65d4-42d0-be1b-0551b5a2f33a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:07.254327 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e958bacd-65d4-42d0-be1b-0551b5a2f33a admin admin] The token cache is being invalidate because project 1690e8e04a3b48ebbe99c45cbe8a28a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:07.255225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e958bacd-65d4-42d0-be1b-0551b5a2f33a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:07.255981 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e958bacd-65d4-42d0-be1b-0551b5a2f33a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:07.258321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 920/1887] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:07 2022] DELETE /identity/v3/projects/1690e8e04a3b48ebbe99c45cbe8a28a2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:07.263828 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c1b7a-28d0-43d7-8bba-f5455f29586d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:07.264391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c1b7a-28d0-43d7-8bba-f5455f29586d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:07.264391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c1b7a-28d0-43d7-8bba-f5455f29586d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:07.284738 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4f9c1b7a-28d0-43d7-8bba-f5455f29586d None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:07.286061 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4f9c1b7a-28d0-43d7-8bba-f5455f29586d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:07.357780 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 968/1888] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:43:07.366996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f94d5-6080-4f97-b6b5-be2cec6967fb None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:07.367286 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f94d5-6080-4f97-b6b5-be2cec6967fb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:07.367570 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f94d5-6080-4f97-b6b5-be2cec6967fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:07.396575 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ce7f94d5-6080-4f97-b6b5-be2cec6967fb None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:07.398168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ce7f94d5-6080-4f97-b6b5-be2cec6967fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:07.419279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 921/1889] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:07.423075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c1b7a-28d0-43d7-8bba-f5455f29586d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:07.446497 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c1b7a-28d0-43d7-8bba-f5455f29586d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:07.463345 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f522c-1a4d-45c1-96c7-94ccb678a489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:07.465437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f522c-1a4d-45c1-96c7-94ccb678a489 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:07.465896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f522c-1a4d-45c1-96c7-94ccb678a489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:07.466414 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f522c-1a4d-45c1-96c7-94ccb678a489 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:07.469338 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f522c-1a4d-45c1-96c7-94ccb678a489 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:43:07.469874 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f522c-1a4d-45c1-96c7-94ccb678a489 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:07.471891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f522c-1a4d-45c1-96c7-94ccb678a489 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:07.488227 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 969/1890] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:43:07.493047 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f94d5-6080-4f97-b6b5-be2cec6967fb None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:07.503143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f94d5-6080-4f97-b6b5-be2cec6967fb None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:07.508921 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86058d6-2e55-4147-9c5f-f0b542642df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:07.510535 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86058d6-2e55-4147-9c5f-f0b542642df3 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:07.510535 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86058d6-2e55-4147-9c5f-f0b542642df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:07.510535 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86058d6-2e55-4147-9c5f-f0b542642df3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:07.511072 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86058d6-2e55-4147-9c5f-f0b542642df3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:07.513245 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86058d6-2e55-4147-9c5f-f0b542642df3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:07.578947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 922/1891] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:07 2022] POST /identity/v3/projects => generated 387 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:07.583527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f522c-1a4d-45c1-96c7-94ccb678a489 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:07.608615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f522c-1a4d-45c1-96c7-94ccb678a489 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:07.694433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9acbed-ec60-4d6b-abc3-13feff0b0d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:07.695203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9acbed-ec60-4d6b-abc3-13feff0b0d06 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:07.695561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9acbed-ec60-4d6b-abc3-13feff0b0d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:07.695913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9acbed-ec60-4d6b-abc3-13feff0b0d06 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:07.696835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9acbed-ec60-4d6b-abc3-13feff0b0d06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:07.698940 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9acbed-ec60-4d6b-abc3-13feff0b0d06 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:07.723245 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 970/1892] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:07 2022] POST /identity/v3/users => generated 294 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:07.728881 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86058d6-2e55-4147-9c5f-f0b542642df3 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:07.753138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86058d6-2e55-4147-9c5f-f0b542642df3 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:07.762229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013065e-538a-422f-8c2d-bdb0ed5c0d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:07.763295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013065e-538a-422f-8c2d-bdb0ed5c0d83 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:07.763788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013065e-538a-422f-8c2d-bdb0ed5c0d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:07.764320 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013065e-538a-422f-8c2d-bdb0ed5c0d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:07.773915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6013065e-538a-422f-8c2d-bdb0ed5c0d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:07.777478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6013065e-538a-422f-8c2d-bdb0ed5c0d83 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:07.828497 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 923/1893] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:07.834726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9acbed-ec60-4d6b-abc3-13feff0b0d06 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:07.851629 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9acbed-ec60-4d6b-abc3-13feff0b0d06 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:07.867815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488f860-6623-41fc-a883-da893d19f32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:07.868678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488f860-6623-41fc-a883-da893d19f32e admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:07.869122 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488f860-6623-41fc-a883-da893d19f32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:07.869612 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488f860-6623-41fc-a883-da893d19f32e admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:07.871744 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488f860-6623-41fc-a883-da893d19f32e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:07.873415 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488f860-6623-41fc-a883-da893d19f32e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:07.895450 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 971/1894] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:07.899661 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013065e-538a-422f-8c2d-bdb0ed5c0d83 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:07.916821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013065e-538a-422f-8c2d-bdb0ed5c0d83 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:07.930262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68415756-c7ca-432a-8e59-52d75b66d7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:07.931483 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68415756-c7ca-432a-8e59-52d75b66d7e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:07.931814 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68415756-c7ca-432a-8e59-52d75b66d7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:07.932153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68415756-c7ca-432a-8e59-52d75b66d7e6 admin admin] PATH_INFO: `/v3/projects/8c0c419fb399434eaf86606b051000cb/users/da4e7f20c4ff446bb15263708775c6d1/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:07.950626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68415756-c7ca-432a-8e59-52d75b66d7e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c0c419fb399434eaf86606b051000cb, user_id=da4e7f20c4ff446bb15263708775c6d1, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:07.952928 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68415756-c7ca-432a-8e59-52d75b66d7e6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:07.968169 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 924/1895] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:07 2022] PUT /identity/v3/projects/8c0c419fb399434eaf86606b051000cb/users/da4e7f20c4ff446bb15263708775c6d1/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:43:07.979527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87450f0d-2f13-4bf6-bf3b-7432aa67f4b7 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:07.979827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87450f0d-2f13-4bf6-bf3b-7432aa67f4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:07.980143 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87450f0d-2f13-4bf6-bf3b-7432aa67f4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:08.003558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:08.016619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-87450f0d-2f13-4bf6-bf3b-7432aa67f4b7 None None] MFA Rules not processed for user `da4e7f20c4ff446bb15263708775c6d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:08.018658 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-87450f0d-2f13-4bf6-bf3b-7432aa67f4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:08.096587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 972/1896] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:08.101274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68415756-c7ca-432a-8e59-52d75b66d7e6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:08.201574 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68415756-c7ca-432a-8e59-52d75b66d7e6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:08.210720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd86373-abc7-49fd-a849-d86b5489bf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:08.212206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd86373-abc7-49fd-a849-d86b5489bf43 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:08.212391 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd86373-abc7-49fd-a849-d86b5489bf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:08.212676 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd86373-abc7-49fd-a849-d86b5489bf43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:08.213526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd86373-abc7-49fd-a849-d86b5489bf43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:08.216494 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd86373-abc7-49fd-a849-d86b5489bf43 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:08.246008 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 925/1897] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:08 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:08.249452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87450f0d-2f13-4bf6-bf3b-7432aa67f4b7 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:08.262117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87450f0d-2f13-4bf6-bf3b-7432aa67f4b7 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:08.347541 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cad2c2-2783-45dd-b36a-2fea30fdd6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:08.349787 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cad2c2-2783-45dd-b36a-2fea30fdd6fa admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:08.350323 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cad2c2-2783-45dd-b36a-2fea30fdd6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:08.350809 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cad2c2-2783-45dd-b36a-2fea30fdd6fa admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:08.351964 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cad2c2-2783-45dd-b36a-2fea30fdd6fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:08.355121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cad2c2-2783-45dd-b36a-2fea30fdd6fa admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:08.382185 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 973/1898] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:08 2022] POST /identity/v3/users => generated 299 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:08.382498 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd86373-abc7-49fd-a849-d86b5489bf43 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:08.393862 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd86373-abc7-49fd-a849-d86b5489bf43 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:08.400846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cdbcc-5136-4cdf-a261-86aad4251fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:08.401717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cdbcc-5136-4cdf-a261-86aad4251fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:08.402051 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cdbcc-5136-4cdf-a261-86aad4251fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:08.402335 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cdbcc-5136-4cdf-a261-86aad4251fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:08.403086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438cdbcc-5136-4cdf-a261-86aad4251fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:08.404735 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438cdbcc-5136-4cdf-a261-86aad4251fb8 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:08.420854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 926/1899] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:08.425082 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cad2c2-2783-45dd-b36a-2fea30fdd6fa admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:08.435730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cad2c2-2783-45dd-b36a-2fea30fdd6fa admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:08.446421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5c217-ae4d-48c4-b643-807e81409d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:08.447368 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5c217-ae4d-48c4-b643-807e81409d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:08.448029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5c217-ae4d-48c4-b643-807e81409d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:08.448309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5c217-ae4d-48c4-b643-807e81409d7b admin admin] PATH_INFO: `/v3/projects/8bae7733e98244d98af7c64dd823ce80/users/114e5a904ad346748f3d811ab38753d2/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:08.467286 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b5c217-ae4d-48c4-b643-807e81409d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bae7733e98244d98af7c64dd823ce80, user_id=114e5a904ad346748f3d811ab38753d2, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:08.470204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b5c217-ae4d-48c4-b643-807e81409d7b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:08.492384 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 974/1900] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:08 2022] PUT /identity/v3/projects/8bae7733e98244d98af7c64dd823ce80/users/114e5a904ad346748f3d811ab38753d2/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:43:08.496295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cdbcc-5136-4cdf-a261-86aad4251fb8 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:08.506813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cdbcc-5136-4cdf-a261-86aad4251fb8 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:08.522479 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:08.581809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65e5b9-1211-4531-9390-eac85b74cbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:08.582574 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65e5b9-1211-4531-9390-eac85b74cbe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:08.582906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65e5b9-1211-4531-9390-eac85b74cbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:08.583673 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65e5b9-1211-4531-9390-eac85b74cbe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:08.584396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65e5b9-1211-4531-9390-eac85b74cbe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:08.586080 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65e5b9-1211-4531-9390-eac85b74cbe4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:08.599369 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 927/1901] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:08.600898 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5c217-ae4d-48c4-b643-807e81409d7b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:08.614155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5c217-ae4d-48c4-b643-807e81409d7b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:08.623902 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c4af7-50e0-4ea0-a443-bef9876297fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:08.625240 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c4af7-50e0-4ea0-a443-bef9876297fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:08.625683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c4af7-50e0-4ea0-a443-bef9876297fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:08.626833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c4af7-50e0-4ea0-a443-bef9876297fc admin admin] PATH_INFO: `/v3/projects/8bae7733e98244d98af7c64dd823ce80/users/114e5a904ad346748f3d811ab38753d2/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:08.630443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c4af7-50e0-4ea0-a443-bef9876297fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bae7733e98244d98af7c64dd823ce80, user_id=114e5a904ad346748f3d811ab38753d2, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:08.634300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c4af7-50e0-4ea0-a443-bef9876297fc admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:08.648930 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 975/1902] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:08 2022] PUT /identity/v3/projects/8bae7733e98244d98af7c64dd823ce80/users/114e5a904ad346748f3d811ab38753d2/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:08.660333 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d6a58-76e4-40ed-b577-b3ced1cdfe9b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:08.660620 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d6a58-76e4-40ed-b577-b3ced1cdfe9b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:08.660904 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d6a58-76e4-40ed-b577-b3ced1cdfe9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:08.691086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e01d6a58-76e4-40ed-b577-b3ced1cdfe9b None None] MFA Rules not processed for user `114e5a904ad346748f3d811ab38753d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:08.693220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e01d6a58-76e4-40ed-b577-b3ced1cdfe9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:08.780889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 928/1903] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:08.795425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c4af7-50e0-4ea0-a443-bef9876297fc admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:08.798413 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-281c4af7-50e0-4ea0-a443-bef9876297fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:08.872367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c4af7-50e0-4ea0-a443-bef9876297fc admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:08.877471 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7645a-75eb-4d0f-89e2-75c89e1393fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:08.878535 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7645a-75eb-4d0f-89e2-75c89e1393fe service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:08.879042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7645a-75eb-4d0f-89e2-75c89e1393fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:08.879602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7645a-75eb-4d0f-89e2-75c89e1393fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:08.895890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:08.900822 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7645a-75eb-4d0f-89e2-75c89e1393fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:08.902877 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7645a-75eb-4d0f-89e2-75c89e1393fe service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:08.927534 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 976/1904] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:08.959602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d6a58-76e4-40ed-b577-b3ced1cdfe9b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:09.042472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d6a58-76e4-40ed-b577-b3ced1cdfe9b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:09.047730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c369e-1309-45d3-83e9-2f6e5a1ee21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:09.048829 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c369e-1309-45d3-83e9-2f6e5a1ee21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:09.049184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c369e-1309-45d3-83e9-2f6e5a1ee21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:09.049522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c369e-1309-45d3-83e9-2f6e5a1ee21d admin admin] PATH_INFO: `/v3/users/da4e7f20c4ff446bb15263708775c6d1` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:09.050908 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c369e-1309-45d3-83e9-2f6e5a1ee21d admin admin] RBAC: Authorizing `identity:delete_user(user_id=da4e7f20c4ff446bb15263708775c6d1)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:09.053005 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c369e-1309-45d3-83e9-2f6e5a1ee21d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:09.105567 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:09.111571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7a0c369e-1309-45d3-83e9-2f6e5a1ee21d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da4e7f20c4ff446bb15263708775c6d1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:09.124563 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7a0c369e-1309-45d3-83e9-2f6e5a1ee21d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da4e7f20c4ff446bb15263708775c6d1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:09.134808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7a0c369e-1309-45d3-83e9-2f6e5a1ee21d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da4e7f20c4ff446bb15263708775c6d1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:09.136033 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7a0c369e-1309-45d3-83e9-2f6e5a1ee21d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da4e7f20c4ff446bb15263708775c6d1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:09.143528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7a0c369e-1309-45d3-83e9-2f6e5a1ee21d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da4e7f20c4ff446bb15263708775c6d1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:09.150490 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 929/1905] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:08 2022] DELETE /identity/v3/users/da4e7f20c4ff446bb15263708775c6d1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:09.155231 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80413294-2403-42a5-9642-f700c32f9acc None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:09.155547 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80413294-2403-42a5-9642-f700c32f9acc None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:09.155872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80413294-2403-42a5-9642-f700c32f9acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:09.174532 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-80413294-2403-42a5-9642-f700c32f9acc None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:09.175748 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-80413294-2403-42a5-9642-f700c32f9acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:09.241622 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 977/1906] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:09.252133 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c369e-1309-45d3-83e9-2f6e5a1ee21d admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:09.253576 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7a0c369e-1309-45d3-83e9-2f6e5a1ee21d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:09.321990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c369e-1309-45d3-83e9-2f6e5a1ee21d admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:09.325908 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568406ea-615d-44a1-9c8f-8caf089c4c51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:09.326630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568406ea-615d-44a1-9c8f-8caf089c4c51 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:09.326912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568406ea-615d-44a1-9c8f-8caf089c4c51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:09.327185 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568406ea-615d-44a1-9c8f-8caf089c4c51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:09.338612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568406ea-615d-44a1-9c8f-8caf089c4c51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:09.340561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568406ea-615d-44a1-9c8f-8caf089c4c51 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:09.352938 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 930/1907] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:09.647768 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80413294-2403-42a5-9642-f700c32f9acc None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:09.649470 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-80413294-2403-42a5-9642-f700c32f9acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:09.673287 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80413294-2403-42a5-9642-f700c32f9acc None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:09.679438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42396789-dfc3-45cb-b50f-bda6927d04f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:09.680579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42396789-dfc3-45cb-b50f-bda6927d04f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:09.680974 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42396789-dfc3-45cb-b50f-bda6927d04f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:09.681376 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42396789-dfc3-45cb-b50f-bda6927d04f9 admin admin] PATH_INFO: `/v3/projects/8c0c419fb399434eaf86606b051000cb` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:09.682706 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42396789-dfc3-45cb-b50f-bda6927d04f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c0c419fb399434eaf86606b051000cb)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:09.684710 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42396789-dfc3-45cb-b50f-bda6927d04f9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:09.738011 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-42396789-dfc3-45cb-b50f-bda6927d04f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c0c419fb399434eaf86606b051000cb', 'request_id': 'req-42396789-dfc3-45cb-b50f-bda6927d04f9'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:09.747100 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-42396789-dfc3-45cb-b50f-bda6927d04f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c0c419fb399434eaf86606b051000cb', 'request_id': 'req-42396789-dfc3-45cb-b50f-bda6927d04f9'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:09.754259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-42396789-dfc3-45cb-b50f-bda6927d04f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c0c419fb399434eaf86606b051000cb', 'request_id': 'req-42396789-dfc3-45cb-b50f-bda6927d04f9'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:09.767781 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-42396789-dfc3-45cb-b50f-bda6927d04f9 admin admin] The token cache is being invalidate because project 8c0c419fb399434eaf86606b051000cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:09.768082 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-42396789-dfc3-45cb-b50f-bda6927d04f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:09.768792 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-42396789-dfc3-45cb-b50f-bda6927d04f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:09.770451 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 978/1908] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:09 2022] DELETE /identity/v3/projects/8c0c419fb399434eaf86606b051000cb => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:09.774724 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568406ea-615d-44a1-9c8f-8caf089c4c51 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:09.785188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568406ea-615d-44a1-9c8f-8caf089c4c51 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:09.846081 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfba88-c3d8-4e28-905f-29b0a48ad84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:09.846995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfba88-c3d8-4e28-905f-29b0a48ad84d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:09.847275 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfba88-c3d8-4e28-905f-29b0a48ad84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:09.847553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfba88-c3d8-4e28-905f-29b0a48ad84d admin admin] PATH_INFO: `/v3/users/114e5a904ad346748f3d811ab38753d2` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:09.848889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfba88-c3d8-4e28-905f-29b0a48ad84d admin admin] RBAC: Authorizing `identity:delete_user(user_id=114e5a904ad346748f3d811ab38753d2)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:09.851105 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfba88-c3d8-4e28-905f-29b0a48ad84d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:09.905442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b0cfba88-c3d8-4e28-905f-29b0a48ad84d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '114e5a904ad346748f3d811ab38753d2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:09.918096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:09.921907 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b0cfba88-c3d8-4e28-905f-29b0a48ad84d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '114e5a904ad346748f3d811ab38753d2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:09.936564 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b0cfba88-c3d8-4e28-905f-29b0a48ad84d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '114e5a904ad346748f3d811ab38753d2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:09.937278 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b0cfba88-c3d8-4e28-905f-29b0a48ad84d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '114e5a904ad346748f3d811ab38753d2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:09.947068 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b0cfba88-c3d8-4e28-905f-29b0a48ad84d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '114e5a904ad346748f3d811ab38753d2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:09.954964 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 931/1909] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:09 2022] DELETE /identity/v3/users/114e5a904ad346748f3d811ab38753d2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:10.334432 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42396789-dfc3-45cb-b50f-bda6927d04f9 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:10.336256 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-42396789-dfc3-45cb-b50f-bda6927d04f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:10.351170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42396789-dfc3-45cb-b50f-bda6927d04f9 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:10.412801 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e8574-e0c5-4e23-82f5-2294d4b2241b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:10.413671 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e8574-e0c5-4e23-82f5-2294d4b2241b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:10.413977 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e8574-e0c5-4e23-82f5-2294d4b2241b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:10.414267 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e8574-e0c5-4e23-82f5-2294d4b2241b admin admin] PATH_INFO: `/v3/projects/8bae7733e98244d98af7c64dd823ce80` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:10.415475 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e8574-e0c5-4e23-82f5-2294d4b2241b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bae7733e98244d98af7c64dd823ce80)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:10.417382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e8574-e0c5-4e23-82f5-2294d4b2241b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:10.441579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:10.468991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d95e8574-e0c5-4e23-82f5-2294d4b2241b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bae7733e98244d98af7c64dd823ce80', 'request_id': 'req-d95e8574-e0c5-4e23-82f5-2294d4b2241b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:10.477998 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d95e8574-e0c5-4e23-82f5-2294d4b2241b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bae7733e98244d98af7c64dd823ce80', 'request_id': 'req-d95e8574-e0c5-4e23-82f5-2294d4b2241b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:10.484827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d95e8574-e0c5-4e23-82f5-2294d4b2241b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bae7733e98244d98af7c64dd823ce80', 'request_id': 'req-d95e8574-e0c5-4e23-82f5-2294d4b2241b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:10.496259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d95e8574-e0c5-4e23-82f5-2294d4b2241b admin admin] The token cache is being invalidate because project 8bae7733e98244d98af7c64dd823ce80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:10.496552 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d95e8574-e0c5-4e23-82f5-2294d4b2241b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:10.497234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d95e8574-e0c5-4e23-82f5-2294d4b2241b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:10.499288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 979/1910] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:10 2022] DELETE /identity/v3/projects/8bae7733e98244d98af7c64dd823ce80 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:10.503930 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8a9b4-0d5c-46f3-8af1-25c5616a6822 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:10.504390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8a9b4-0d5c-46f3-8af1-25c5616a6822 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:10.504501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8a9b4-0d5c-46f3-8af1-25c5616a6822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:10.524122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-39b8a9b4-0d5c-46f3-8af1-25c5616a6822 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:10.525337 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-39b8a9b4-0d5c-46f3-8af1-25c5616a6822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:10.588837 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 932/1911] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:10.596460 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c80ff6-bf42-41a2-aab8-cca57f231c73 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:10.596753 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c80ff6-bf42-41a2-aab8-cca57f231c73 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:10.597032 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c80ff6-bf42-41a2-aab8-cca57f231c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:10.615964 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c2c80ff6-bf42-41a2-aab8-cca57f231c73 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:10.617200 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c2c80ff6-bf42-41a2-aab8-cca57f231c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:10.625349 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 980/1912] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:10.630102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8a9b4-0d5c-46f3-8af1-25c5616a6822 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:10.640248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8a9b4-0d5c-46f3-8af1-25c5616a6822 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:10.644880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f140a-7f37-48ea-ab31-f5f3a14b8c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:10.646252 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f140a-7f37-48ea-ab31-f5f3a14b8c44 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:10.646531 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f140a-7f37-48ea-ab31-f5f3a14b8c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:10.646800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f140a-7f37-48ea-ab31-f5f3a14b8c44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:10.647524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f140a-7f37-48ea-ab31-f5f3a14b8c44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:43:10.647823 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f140a-7f37-48ea-ab31-f5f3a14b8c44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:10.649404 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f140a-7f37-48ea-ab31-f5f3a14b8c44 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:10.660738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 933/1913] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:43:10.665532 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c80ff6-bf42-41a2-aab8-cca57f231c73 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:10.675186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c80ff6-bf42-41a2-aab8-cca57f231c73 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:10.680378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae471ce5-302a-4d47-8507-78f34e60aae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:10.681038 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae471ce5-302a-4d47-8507-78f34e60aae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:10.681323 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae471ce5-302a-4d47-8507-78f34e60aae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:10.681603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae471ce5-302a-4d47-8507-78f34e60aae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:10.682355 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae471ce5-302a-4d47-8507-78f34e60aae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:10.684326 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae471ce5-302a-4d47-8507-78f34e60aae4 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:10.710542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 981/1914] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:10 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:43:10.714592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f140a-7f37-48ea-ab31-f5f3a14b8c44 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:10.725221 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f140a-7f37-48ea-ab31-f5f3a14b8c44 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:10.780638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c53c8-a213-4f60-9bb5-b2f0f3cb69bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:10.781307 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c53c8-a213-4f60-9bb5-b2f0f3cb69bb admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:10.781592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c53c8-a213-4f60-9bb5-b2f0f3cb69bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:10.781867 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c53c8-a213-4f60-9bb5-b2f0f3cb69bb admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:10.782591 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c53c8-a213-4f60-9bb5-b2f0f3cb69bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:10.784501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c53c8-a213-4f60-9bb5-b2f0f3cb69bb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:10.804816 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 934/1915] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:10 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:43:10.808974 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae471ce5-302a-4d47-8507-78f34e60aae4 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:10.819836 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae471ce5-302a-4d47-8507-78f34e60aae4 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:10.824519 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d7d1b-02b0-48b5-9680-187071ddf936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:10.825684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d7d1b-02b0-48b5-9680-187071ddf936 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:10.826004 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d7d1b-02b0-48b5-9680-187071ddf936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:10.826704 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d7d1b-02b0-48b5-9680-187071ddf936 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:10.827444 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d7d1b-02b0-48b5-9680-187071ddf936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:10.829044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d7d1b-02b0-48b5-9680-187071ddf936 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:10.842452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 982/1916] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:10.846619 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c53c8-a213-4f60-9bb5-b2f0f3cb69bb admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:10.858353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c53c8-a213-4f60-9bb5-b2f0f3cb69bb admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:10.863031 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8a78f-4e8d-4b5b-a7ad-b79213fe4d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:10.863815 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8a78f-4e8d-4b5b-a7ad-b79213fe4d5c admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:10.864142 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8a78f-4e8d-4b5b-a7ad-b79213fe4d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:10.864419 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8a78f-4e8d-4b5b-a7ad-b79213fe4d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:10.865149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8a78f-4e8d-4b5b-a7ad-b79213fe4d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:10.866791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8a78f-4e8d-4b5b-a7ad-b79213fe4d5c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:10.877907 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 935/1917] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:43:10.882541 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d7d1b-02b0-48b5-9680-187071ddf936 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:10.892897 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d7d1b-02b0-48b5-9680-187071ddf936 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:10.898326 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78801086-013c-4c48-bb84-15c3dd5be9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:10.899235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78801086-013c-4c48-bb84-15c3dd5be9e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:10.899523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78801086-013c-4c48-bb84-15c3dd5be9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:10.899804 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78801086-013c-4c48-bb84-15c3dd5be9e6 admin admin] PATH_INFO: `/v3/projects/e1aa729ed0af42c08d12727192374922/users/2178c06e1f994a5da7ac25c6434256ea/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:10.915273 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78801086-013c-4c48-bb84-15c3dd5be9e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1aa729ed0af42c08d12727192374922, user_id=2178c06e1f994a5da7ac25c6434256ea, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:10.917949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78801086-013c-4c48-bb84-15c3dd5be9e6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:10.931707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 983/1918] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:10 2022] PUT /identity/v3/projects/e1aa729ed0af42c08d12727192374922/users/2178c06e1f994a5da7ac25c6434256ea/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:10.941898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f5530-de18-4d44-ac76-44500d377dc6 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:10.942734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f5530-de18-4d44-ac76-44500d377dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:10.943017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f5530-de18-4d44-ac76-44500d377dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:10.971605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-314f5530-de18-4d44-ac76-44500d377dc6 None None] MFA Rules not processed for user `2178c06e1f994a5da7ac25c6434256ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:10.973763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-314f5530-de18-4d44-ac76-44500d377dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:11.012637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:11.049437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 936/1919] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:11.054058 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78801086-013c-4c48-bb84-15c3dd5be9e6 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:11.146226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78801086-013c-4c48-bb84-15c3dd5be9e6 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:11.152308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec32f2-1fc7-4a4c-9335-976f7a56d02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:11.153108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec32f2-1fc7-4a4c-9335-976f7a56d02a admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:11.153462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec32f2-1fc7-4a4c-9335-976f7a56d02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:11.153807 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec32f2-1fc7-4a4c-9335-976f7a56d02a admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:11.154666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec32f2-1fc7-4a4c-9335-976f7a56d02a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:11.156985 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec32f2-1fc7-4a4c-9335-976f7a56d02a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:11.190127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 984/1920] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:11 2022] POST /identity/v3/projects => generated 381 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:43:11.191222 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f5530-de18-4d44-ac76-44500d377dc6 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:11.202761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f5530-de18-4d44-ac76-44500d377dc6 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:11.265624 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae26f8-78a6-44cb-8117-b440ef69f804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:11.266342 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae26f8-78a6-44cb-8117-b440ef69f804 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:11.266630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae26f8-78a6-44cb-8117-b440ef69f804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:11.266909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae26f8-78a6-44cb-8117-b440ef69f804 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:11.267619 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae26f8-78a6-44cb-8117-b440ef69f804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:11.269536 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae26f8-78a6-44cb-8117-b440ef69f804 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:11.298712 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 937/1921] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:11 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:43:11.302851 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec32f2-1fc7-4a4c-9335-976f7a56d02a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:11.314408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec32f2-1fc7-4a4c-9335-976f7a56d02a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:11.320491 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190419ab-8eb4-49d1-8280-3011753dc9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:11.322051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190419ab-8eb4-49d1-8280-3011753dc9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:11.322368 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190419ab-8eb4-49d1-8280-3011753dc9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:11.322717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190419ab-8eb4-49d1-8280-3011753dc9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:11.323519 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190419ab-8eb4-49d1-8280-3011753dc9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:11.325339 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190419ab-8eb4-49d1-8280-3011753dc9f3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:11.336712 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 985/1922] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:11.340655 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae26f8-78a6-44cb-8117-b440ef69f804 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:11.350647 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae26f8-78a6-44cb-8117-b440ef69f804 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:11.355242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88406a-c219-47dd-a0dd-518dc86bbb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:11.356120 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88406a-c219-47dd-a0dd-518dc86bbb73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:11.356406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88406a-c219-47dd-a0dd-518dc86bbb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:11.356687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88406a-c219-47dd-a0dd-518dc86bbb73 admin admin] PATH_INFO: `/v3/projects/e18f275beadd44b6b13cd2cd64261299/users/8d9d99fff9c24eb5bca9c9ac21c82ada/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:11.372026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88406a-c219-47dd-a0dd-518dc86bbb73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18f275beadd44b6b13cd2cd64261299, user_id=8d9d99fff9c24eb5bca9c9ac21c82ada, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:11.374718 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88406a-c219-47dd-a0dd-518dc86bbb73 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:11.388349 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 938/1923] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:11 2022] PUT /identity/v3/projects/e18f275beadd44b6b13cd2cd64261299/users/8d9d99fff9c24eb5bca9c9ac21c82ada/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:11.392037 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190419ab-8eb4-49d1-8280-3011753dc9f3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:11.401986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190419ab-8eb4-49d1-8280-3011753dc9f3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:11.417879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:11.459230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cdb1d-6ec8-476d-ade5-efe3afc79c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:11.459909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cdb1d-6ec8-476d-ade5-efe3afc79c6b admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:11.460198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cdb1d-6ec8-476d-ade5-efe3afc79c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:11.460476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cdb1d-6ec8-476d-ade5-efe3afc79c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:11.461198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cdb1d-6ec8-476d-ade5-efe3afc79c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:11.462864 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cdb1d-6ec8-476d-ade5-efe3afc79c6b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:11.474767 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 986/1924] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:11.477721 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88406a-c219-47dd-a0dd-518dc86bbb73 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:11.488148 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88406a-c219-47dd-a0dd-518dc86bbb73 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:11.493331 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a62716-15b8-4227-b887-611e90458ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:11.494246 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a62716-15b8-4227-b887-611e90458ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:11.494532 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a62716-15b8-4227-b887-611e90458ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:11.494811 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a62716-15b8-4227-b887-611e90458ce7 admin admin] PATH_INFO: `/v3/projects/e18f275beadd44b6b13cd2cd64261299/users/8d9d99fff9c24eb5bca9c9ac21c82ada/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:11.496801 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a62716-15b8-4227-b887-611e90458ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18f275beadd44b6b13cd2cd64261299, user_id=8d9d99fff9c24eb5bca9c9ac21c82ada, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:11.499464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a62716-15b8-4227-b887-611e90458ce7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:11.510962 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 939/1925] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:11 2022] PUT /identity/v3/projects/e18f275beadd44b6b13cd2cd64261299/users/8d9d99fff9c24eb5bca9c9ac21c82ada/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:11.522111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a56a83-23ea-4dde-85d2-96930c697244 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:11.522414 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a56a83-23ea-4dde-85d2-96930c697244 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:11.522695 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a56a83-23ea-4dde-85d2-96930c697244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:11.549294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c3a56a83-23ea-4dde-85d2-96930c697244 None None] MFA Rules not processed for user `8d9d99fff9c24eb5bca9c9ac21c82ada`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:11.550859 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c3a56a83-23ea-4dde-85d2-96930c697244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:11.637108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 987/1926] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:11.647920 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a62716-15b8-4227-b887-611e90458ce7 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:11.649763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-10a62716-15b8-4227-b887-611e90458ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:11.722469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a62716-15b8-4227-b887-611e90458ce7 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:11.734180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4934a09-c441-42a1-a423-e1ef77945848 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:11.734180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4934a09-c441-42a1-a423-e1ef77945848 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:11.734180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4934a09-c441-42a1-a423-e1ef77945848 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:11.734180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4934a09-c441-42a1-a423-e1ef77945848 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:11.751090 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4934a09-c441-42a1-a423-e1ef77945848 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:11.753645 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4934a09-c441-42a1-a423-e1ef77945848 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:11.779918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 940/1927] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:11.947839 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a56a83-23ea-4dde-85d2-96930c697244 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:11.967558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a56a83-23ea-4dde-85d2-96930c697244 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:11.973097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df315f7-2a2b-4e1a-81f1-7848a9449757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:11.974985 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df315f7-2a2b-4e1a-81f1-7848a9449757 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:11.975485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df315f7-2a2b-4e1a-81f1-7848a9449757 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:11.976054 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df315f7-2a2b-4e1a-81f1-7848a9449757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:12.042703 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df315f7-2a2b-4e1a-81f1-7848a9449757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:12.044259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df315f7-2a2b-4e1a-81f1-7848a9449757 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:12.055765 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 988/1928] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:14.127590 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4934a09-c441-42a1-a423-e1ef77945848 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:14.129081 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f4934a09-c441-42a1-a423-e1ef77945848 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:14.166277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:14.202696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4934a09-c441-42a1-a423-e1ef77945848 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:14.207463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc8f45-553e-4d9b-bb10-0eda831d8de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:14.208371 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc8f45-553e-4d9b-bb10-0eda831d8de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:14.208657 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc8f45-553e-4d9b-bb10-0eda831d8de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:14.208937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc8f45-553e-4d9b-bb10-0eda831d8de1 admin admin] PATH_INFO: `/v3/users/b5388e1730274b08bbeb8dbed7fad5d2` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:14.210217 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc8f45-553e-4d9b-bb10-0eda831d8de1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5388e1730274b08bbeb8dbed7fad5d2)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:14.212225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc8f45-553e-4d9b-bb10-0eda831d8de1 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:14.261649 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-fcdc8f45-553e-4d9b-bb10-0eda831d8de1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5388e1730274b08bbeb8dbed7fad5d2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:14.275374 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-fcdc8f45-553e-4d9b-bb10-0eda831d8de1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5388e1730274b08bbeb8dbed7fad5d2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:14.284157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-fcdc8f45-553e-4d9b-bb10-0eda831d8de1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5388e1730274b08bbeb8dbed7fad5d2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:14.285001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-fcdc8f45-553e-4d9b-bb10-0eda831d8de1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5388e1730274b08bbeb8dbed7fad5d2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:14.293211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-fcdc8f45-553e-4d9b-bb10-0eda831d8de1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5388e1730274b08bbeb8dbed7fad5d2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:14.303317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 941/1929] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:14 2022] DELETE /identity/v3/users/b5388e1730274b08bbeb8dbed7fad5d2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:14.306771 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f9614-0230-463b-9045-94f094c72442 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:14.307241 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f9614-0230-463b-9045-94f094c72442 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:14.307511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f9614-0230-463b-9045-94f094c72442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:14.327992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9e8f9614-0230-463b-9045-94f094c72442 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:14.329219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9e8f9614-0230-463b-9045-94f094c72442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:14.362637 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:14.392053 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 989/1930] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:14.402115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc8f45-553e-4d9b-bb10-0eda831d8de1 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:14.403498 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fcdc8f45-553e-4d9b-bb10-0eda831d8de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:14.471019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc8f45-553e-4d9b-bb10-0eda831d8de1 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:14.474841 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf44300-6e83-4ef1-bc97-ea49fa149a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:14.475486 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf44300-6e83-4ef1-bc97-ea49fa149a39 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:14.475805 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf44300-6e83-4ef1-bc97-ea49fa149a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:14.476085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf44300-6e83-4ef1-bc97-ea49fa149a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:14.485814 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf44300-6e83-4ef1-bc97-ea49fa149a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:14.487306 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf44300-6e83-4ef1-bc97-ea49fa149a39 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:14.497494 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 942/1931] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:14.752840 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f9614-0230-463b-9045-94f094c72442 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:14.754282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9e8f9614-0230-463b-9045-94f094c72442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:14.766123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f9614-0230-463b-9045-94f094c72442 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:14.771170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006cb75f-6c0a-43ab-8601-ad10a962ad39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:14.772047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cb75f-6c0a-43ab-8601-ad10a962ad39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:14.772336 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cb75f-6c0a-43ab-8601-ad10a962ad39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:14.772616 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cb75f-6c0a-43ab-8601-ad10a962ad39 admin admin] PATH_INFO: `/v3/users/2178c06e1f994a5da7ac25c6434256ea` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:14.773808 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006cb75f-6c0a-43ab-8601-ad10a962ad39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2178c06e1f994a5da7ac25c6434256ea)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:14.775791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006cb75f-6c0a-43ab-8601-ad10a962ad39 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:14.831236 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf44300-6e83-4ef1-bc97-ea49fa149a39 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:14.831596 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-006cb75f-6c0a-43ab-8601-ad10a962ad39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2178c06e1f994a5da7ac25c6434256ea'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:14.832441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-006cb75f-6c0a-43ab-8601-ad10a962ad39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2178c06e1f994a5da7ac25c6434256ea'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:14.832724 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6cf44300-6e83-4ef1-bc97-ea49fa149a39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:14.833518 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-006cb75f-6c0a-43ab-8601-ad10a962ad39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2178c06e1f994a5da7ac25c6434256ea'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:14.851788 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-006cb75f-6c0a-43ab-8601-ad10a962ad39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2178c06e1f994a5da7ac25c6434256ea'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:14.862852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-006cb75f-6c0a-43ab-8601-ad10a962ad39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2178c06e1f994a5da7ac25c6434256ea'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:14.879039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 990/1932] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:14 2022] DELETE /identity/v3/users/2178c06e1f994a5da7ac25c6434256ea => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:14.884175 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ae41b-5598-4f12-922a-e110a1049c42 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:14.884440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ae41b-5598-4f12-922a-e110a1049c42 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:14.884726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ae41b-5598-4f12-922a-e110a1049c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:14.885026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf44300-6e83-4ef1-bc97-ea49fa149a39 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:14.906767 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4a4ae41b-5598-4f12-922a-e110a1049c42 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:14.907980 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4a4ae41b-5598-4f12-922a-e110a1049c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:14.937942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:14.949232 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de56894-8ae7-4cb5-adc4-cb90981f525f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:14.950062 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de56894-8ae7-4cb5-adc4-cb90981f525f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:14.950594 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de56894-8ae7-4cb5-adc4-cb90981f525f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:14.950937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de56894-8ae7-4cb5-adc4-cb90981f525f admin admin] PATH_INFO: `/v3/projects/64aa4006876542738b9594f1ffefc6fe` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:14.952201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de56894-8ae7-4cb5-adc4-cb90981f525f admin admin] RBAC: Authorizing `identity:delete_project(project_id=64aa4006876542738b9594f1ffefc6fe)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:14.954242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de56894-8ae7-4cb5-adc4-cb90981f525f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:14.969021 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 991/1933] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:43:14.979891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ae41b-5598-4f12-922a-e110a1049c42 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:14.981335 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4a4ae41b-5598-4f12-922a-e110a1049c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:15.005246 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1de56894-8ae7-4cb5-adc4-cb90981f525f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64aa4006876542738b9594f1ffefc6fe', 'request_id': 'req-1de56894-8ae7-4cb5-adc4-cb90981f525f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:15.011757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1de56894-8ae7-4cb5-adc4-cb90981f525f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64aa4006876542738b9594f1ffefc6fe', 'request_id': 'req-1de56894-8ae7-4cb5-adc4-cb90981f525f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:15.019239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1de56894-8ae7-4cb5-adc4-cb90981f525f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64aa4006876542738b9594f1ffefc6fe', 'request_id': 'req-1de56894-8ae7-4cb5-adc4-cb90981f525f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:15.032914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1de56894-8ae7-4cb5-adc4-cb90981f525f admin admin] The token cache is being invalidate because project 64aa4006876542738b9594f1ffefc6fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:15.033210 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1de56894-8ae7-4cb5-adc4-cb90981f525f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:15.033831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1de56894-8ae7-4cb5-adc4-cb90981f525f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:15.035627 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 943/1934] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:14 2022] DELETE /identity/v3/projects/64aa4006876542738b9594f1ffefc6fe => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:43:15.040975 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e709647-67e5-4047-a317-691132a79714 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:15.041259 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e709647-67e5-4047-a317-691132a79714 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:15.041565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e709647-67e5-4047-a317-691132a79714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:15.041855 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ae41b-5598-4f12-922a-e110a1049c42 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:15.046452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:15.059786 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-6e709647-67e5-4047-a317-691132a79714 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:15.060991 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6e709647-67e5-4047-a317-691132a79714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:15.092094 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199a60c-965f-423c-add5-4f51de9c9b4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:15.092857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199a60c-965f-423c-add5-4f51de9c9b4e service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:15.093178 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199a60c-965f-423c-add5-4f51de9c9b4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:15.093652 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199a60c-965f-423c-add5-4f51de9c9b4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:15.095509 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e199a60c-965f-423c-add5-4f51de9c9b4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:15.120538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 944/1935] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:15.128263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abbd66-ddea-4952-a4f3-7b68b584de2b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:15.128876 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abbd66-ddea-4952-a4f3-7b68b584de2b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:15.128876 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abbd66-ddea-4952-a4f3-7b68b584de2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:15.146473 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-52abbd66-ddea-4952-a4f3-7b68b584de2b None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:15.147577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-52abbd66-ddea-4952-a4f3-7b68b584de2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:15.155755 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 945/1936] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:43:15.159621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199a60c-965f-423c-add5-4f51de9c9b4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:15.159902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abbd66-ddea-4952-a4f3-7b68b584de2b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:15.161640 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199a60c-965f-423c-add5-4f51de9c9b4e service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:15.169423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abbd66-ddea-4952-a4f3-7b68b584de2b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:15.171393 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 992/1937] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:43:15.174276 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846a559-a046-40ed-8f94-0b07ca8506e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:15.174917 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846a559-a046-40ed-8f94-0b07ca8506e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:15.175201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846a559-a046-40ed-8f94-0b07ca8506e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:15.175479 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846a559-a046-40ed-8f94-0b07ca8506e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:15.176201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846a559-a046-40ed-8f94-0b07ca8506e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:43:15.176542 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846a559-a046-40ed-8f94-0b07ca8506e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:15.178181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846a559-a046-40ed-8f94-0b07ca8506e4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:15.193750 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 946/1938] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:43:15.195048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199a60c-965f-423c-add5-4f51de9c9b4e service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:15.204678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199a60c-965f-423c-add5-4f51de9c9b4e service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:15.209868 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692513d-fdb2-480d-abc4-c177a65b226f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:15.210567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692513d-fdb2-480d-abc4-c177a65b226f admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:15.210852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692513d-fdb2-480d-abc4-c177a65b226f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:15.211130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692513d-fdb2-480d-abc4-c177a65b226f admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:15.211819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692513d-fdb2-480d-abc4-c177a65b226f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:15.213729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692513d-fdb2-480d-abc4-c177a65b226f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:15.241727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 993/1939] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:15 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:15.246301 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846a559-a046-40ed-8f94-0b07ca8506e4 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:15.257606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846a559-a046-40ed-8f94-0b07ca8506e4 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:15.321011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2998a48-696a-466c-9770-c2b04af9285e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:15.321879 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2998a48-696a-466c-9770-c2b04af9285e admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:15.322195 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2998a48-696a-466c-9770-c2b04af9285e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:15.322469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2998a48-696a-466c-9770-c2b04af9285e admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:15.323276 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2998a48-696a-466c-9770-c2b04af9285e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:15.325485 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2998a48-696a-466c-9770-c2b04af9285e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:15.337581 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:15.354011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 947/1940] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:15 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:43:15.358073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692513d-fdb2-480d-abc4-c177a65b226f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:15.368986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692513d-fdb2-480d-abc4-c177a65b226f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:15.374261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc641c8-0a90-43ee-92ec-dfb9a4ed44b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:15.375497 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc641c8-0a90-43ee-92ec-dfb9a4ed44b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:15.375497 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc641c8-0a90-43ee-92ec-dfb9a4ed44b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:15.375497 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc641c8-0a90-43ee-92ec-dfb9a4ed44b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:15.376294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc641c8-0a90-43ee-92ec-dfb9a4ed44b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:15.378331 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc641c8-0a90-43ee-92ec-dfb9a4ed44b8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:15.393473 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 994/1941] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:15.395068 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2998a48-696a-466c-9770-c2b04af9285e admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:15.405867 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2998a48-696a-466c-9770-c2b04af9285e admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:15.410842 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3ff2c-4016-4a2b-956c-18945d2392d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:15.411505 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3ff2c-4016-4a2b-956c-18945d2392d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:15.411793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3ff2c-4016-4a2b-956c-18945d2392d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:15.412095 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3ff2c-4016-4a2b-956c-18945d2392d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:15.412796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3ff2c-4016-4a2b-956c-18945d2392d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:15.414432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3ff2c-4016-4a2b-956c-18945d2392d9 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:15.424589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 948/1942] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:15.428929 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc641c8-0a90-43ee-92ec-dfb9a4ed44b8 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:15.438824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc641c8-0a90-43ee-92ec-dfb9a4ed44b8 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:15.444908 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b621f6-ec64-4c64-9f03-193985cb3b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:15.445919 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b621f6-ec64-4c64-9f03-193985cb3b16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:15.446237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b621f6-ec64-4c64-9f03-193985cb3b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:15.446517 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b621f6-ec64-4c64-9f03-193985cb3b16 admin admin] PATH_INFO: `/v3/projects/edae8cd268da40a3b8e2affe277bc56e/users/bb11db914a57439bada24b5f755a143a/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:15.464119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b621f6-ec64-4c64-9f03-193985cb3b16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edae8cd268da40a3b8e2affe277bc56e, user_id=bb11db914a57439bada24b5f755a143a, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:15.467301 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3ff2c-4016-4a2b-956c-18945d2392d9 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:15.468674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b621f6-ec64-4c64-9f03-193985cb3b16 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:15.470840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-42c3ff2c-4016-4a2b-956c-18945d2392d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:15.491471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3ff2c-4016-4a2b-956c-18945d2392d9 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:15.498509 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 995/1943] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:15 2022] PUT /identity/v3/projects/edae8cd268da40a3b8e2affe277bc56e/users/bb11db914a57439bada24b5f755a143a/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:15.507901 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49540715-3561-4969-befd-cf9c9da3e642 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:15.508198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49540715-3561-4969-befd-cf9c9da3e642 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:15.508480 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49540715-3561-4969-befd-cf9c9da3e642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:15.528310 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-49540715-3561-4969-befd-cf9c9da3e642 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:15.528310 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-49540715-3561-4969-befd-cf9c9da3e642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:15.574282 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:15.579706 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:15.579706 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:15.579706 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5 admin admin] PATH_INFO: `/v3/projects/e1aa729ed0af42c08d12727192374922` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:15.579706 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1aa729ed0af42c08d12727192374922)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:15.582139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:15.603662 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 996/1944] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:43:15.613128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49540715-3561-4969-befd-cf9c9da3e642 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:15.616282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-49540715-3561-4969-befd-cf9c9da3e642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:15.623543 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:15.644960 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1aa729ed0af42c08d12727192374922', 'request_id': 'req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:15.652088 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1aa729ed0af42c08d12727192374922', 'request_id': 'req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:15.660815 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1aa729ed0af42c08d12727192374922', 'request_id': 'req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:15.674885 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5 admin admin] The token cache is being invalidate because project e1aa729ed0af42c08d12727192374922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:15.675178 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:15.675921 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:15.681554 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 949/1945] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:15 2022] DELETE /identity/v3/projects/e1aa729ed0af42c08d12727192374922 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:15.682466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:15.683909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:15.685410 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49540715-3561-4969-befd-cf9c9da3e642 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:15.742212 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f68a3c-aa25-45e2-86c4-c57c8162058f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:15.742870 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f68a3c-aa25-45e2-86c4-c57c8162058f service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:15.743161 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f68a3c-aa25-45e2-86c4-c57c8162058f service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:15.743457 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f68a3c-aa25-45e2-86c4-c57c8162058f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:15.745472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f4f68a3c-aa25-45e2-86c4-c57c8162058f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:15.755720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee6dd1-29c2-4b9b-8257-39a2db583dc5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:15.760821 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f68a3c-aa25-45e2-86c4-c57c8162058f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:15.762334 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f68a3c-aa25-45e2-86c4-c57c8162058f service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:15.762442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455907cb-f2e9-4878-9ae0-7020fcbadc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:15.763867 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455907cb-f2e9-4878-9ae0-7020fcbadc7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:15.764358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455907cb-f2e9-4878-9ae0-7020fcbadc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:15.765126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455907cb-f2e9-4878-9ae0-7020fcbadc7f admin admin] PATH_INFO: `/v3/users/8d9d99fff9c24eb5bca9c9ac21c82ada` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:15.766711 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455907cb-f2e9-4878-9ae0-7020fcbadc7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d9d99fff9c24eb5bca9c9ac21c82ada)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:15.769928 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455907cb-f2e9-4878-9ae0-7020fcbadc7f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:15.773207 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 997/1946] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:15.789211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:INSERT=2,keystone:SELECT=90,keystone:DELETE=8 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:15.830859 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-455907cb-f2e9-4878-9ae0-7020fcbadc7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d9d99fff9c24eb5bca9c9ac21c82ada'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:15.843967 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-455907cb-f2e9-4878-9ae0-7020fcbadc7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d9d99fff9c24eb5bca9c9ac21c82ada'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:15.861934 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-455907cb-f2e9-4878-9ae0-7020fcbadc7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d9d99fff9c24eb5bca9c9ac21c82ada'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:15.863571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-455907cb-f2e9-4878-9ae0-7020fcbadc7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d9d99fff9c24eb5bca9c9ac21c82ada'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:15.873693 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-455907cb-f2e9-4878-9ae0-7020fcbadc7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d9d99fff9c24eb5bca9c9ac21c82ada'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:15.886362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 950/1947] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:15 2022] DELETE /identity/v3/users/8d9d99fff9c24eb5bca9c9ac21c82ada => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:43:16.574932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f68a3c-aa25-45e2-86c4-c57c8162058f service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:16.600662 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f68a3c-aa25-45e2-86c4-c57c8162058f service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:16.675942 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3dd28-fb21-4526-b451-2bbe6ff64276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:16.677155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3dd28-fb21-4526-b451-2bbe6ff64276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:16.677552 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3dd28-fb21-4526-b451-2bbe6ff64276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:16.677837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3dd28-fb21-4526-b451-2bbe6ff64276 admin admin] PATH_INFO: `/v3/projects/e18f275beadd44b6b13cd2cd64261299` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:16.679101 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3dd28-fb21-4526-b451-2bbe6ff64276 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e18f275beadd44b6b13cd2cd64261299)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:16.681435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3dd28-fb21-4526-b451-2bbe6ff64276 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:16.788875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5ef3dd28-fb21-4526-b451-2bbe6ff64276 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e18f275beadd44b6b13cd2cd64261299', 'request_id': 'req-5ef3dd28-fb21-4526-b451-2bbe6ff64276'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:16.799215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5ef3dd28-fb21-4526-b451-2bbe6ff64276 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e18f275beadd44b6b13cd2cd64261299', 'request_id': 'req-5ef3dd28-fb21-4526-b451-2bbe6ff64276'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:16.813508 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5ef3dd28-fb21-4526-b451-2bbe6ff64276 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e18f275beadd44b6b13cd2cd64261299', 'request_id': 'req-5ef3dd28-fb21-4526-b451-2bbe6ff64276'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:16.817726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:16.838904 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5ef3dd28-fb21-4526-b451-2bbe6ff64276 admin admin] The token cache is being invalidate because project e18f275beadd44b6b13cd2cd64261299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:16.839528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5ef3dd28-fb21-4526-b451-2bbe6ff64276 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:16.840448 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5ef3dd28-fb21-4526-b451-2bbe6ff64276 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:16.850250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 998/1948] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:16 2022] DELETE /identity/v3/projects/e18f275beadd44b6b13cd2cd64261299 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:16.855821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade8dbf-8034-4601-8291-652e343c923b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:16.856294 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade8dbf-8034-4601-8291-652e343c923b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:16.856833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade8dbf-8034-4601-8291-652e343c923b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:16.909660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-eade8dbf-8034-4601-8291-652e343c923b None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:16.911196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-eade8dbf-8034-4601-8291-652e343c923b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:17.006441 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 951/1949] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:17.014803 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d46d2-6bc7-407a-9631-b9bfd0e04320 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:17.015169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d46d2-6bc7-407a-9631-b9bfd0e04320 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:17.015529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d46d2-6bc7-407a-9631-b9bfd0e04320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:17.040471 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-176d46d2-6bc7-407a-9631-b9bfd0e04320 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:17.041984 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-176d46d2-6bc7-407a-9631-b9bfd0e04320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:17.063486 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade8dbf-8034-4601-8291-652e343c923b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:17.075737 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 999/1950] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:17.078691 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade8dbf-8034-4601-8291-652e343c923b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:17.088088 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7701bc-e70b-4ecd-beee-2abe76a0f7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:17.089619 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7701bc-e70b-4ecd-beee-2abe76a0f7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:17.090040 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7701bc-e70b-4ecd-beee-2abe76a0f7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:17.090398 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7701bc-e70b-4ecd-beee-2abe76a0f7b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:17.092134 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7701bc-e70b-4ecd-beee-2abe76a0f7b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:43:17.092650 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7701bc-e70b-4ecd-beee-2abe76a0f7b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:17.095213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7701bc-e70b-4ecd-beee-2abe76a0f7b1 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:17.121280 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d46d2-6bc7-407a-9631-b9bfd0e04320 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:17.127818 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 952/1951] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:43:17.132724 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d46d2-6bc7-407a-9631-b9bfd0e04320 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:17.138365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4b5c4-ddde-4369-ab97-821fa8acd779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:17.139293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4b5c4-ddde-4369-ab97-821fa8acd779 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:17.139592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4b5c4-ddde-4369-ab97-821fa8acd779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:17.140322 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4b5c4-ddde-4369-ab97-821fa8acd779 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:17.141234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4b5c4-ddde-4369-ab97-821fa8acd779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:17.143474 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4b5c4-ddde-4369-ab97-821fa8acd779 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:17.191711 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7701bc-e70b-4ecd-beee-2abe76a0f7b1 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:17.193244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1000/1952] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:17 2022] POST /identity/v3/projects => generated 367 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:17.217129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7701bc-e70b-4ecd-beee-2abe76a0f7b1 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:17.290025 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22129632-3721-4e09-971c-986f344f0809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:17.290683 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22129632-3721-4e09-971c-986f344f0809 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:17.290972 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22129632-3721-4e09-971c-986f344f0809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:17.291254 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22129632-3721-4e09-971c-986f344f0809 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:17.291954 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22129632-3721-4e09-971c-986f344f0809 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:17.293922 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22129632-3721-4e09-971c-986f344f0809 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:17.321296 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 953/1953] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:17 2022] POST /identity/v3/users => generated 284 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:17.325065 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4b5c4-ddde-4369-ab97-821fa8acd779 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:17.340125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4b5c4-ddde-4369-ab97-821fa8acd779 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:17.345590 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e7b5b-022f-4f3a-8d0e-04654b8f8ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:17.346500 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e7b5b-022f-4f3a-8d0e-04654b8f8ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:17.346813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e7b5b-022f-4f3a-8d0e-04654b8f8ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:17.347101 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e7b5b-022f-4f3a-8d0e-04654b8f8ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:17.347912 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e7b5b-022f-4f3a-8d0e-04654b8f8ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:17.350644 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e7b5b-022f-4f3a-8d0e-04654b8f8ea2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:17.384695 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1001/1954] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:17.388285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22129632-3721-4e09-971c-986f344f0809 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:17.400609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22129632-3721-4e09-971c-986f344f0809 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:17.405807 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37b578-0d56-46be-a190-3469e19371ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:17.406560 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37b578-0d56-46be-a190-3469e19371ef admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:17.406852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37b578-0d56-46be-a190-3469e19371ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:17.407131 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37b578-0d56-46be-a190-3469e19371ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:17.408115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc37b578-0d56-46be-a190-3469e19371ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:17.409883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc37b578-0d56-46be-a190-3469e19371ef admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:17.417791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:17.435062 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 954/1955] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:17.436863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e7b5b-022f-4f3a-8d0e-04654b8f8ea2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:17.452711 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e7b5b-022f-4f3a-8d0e-04654b8f8ea2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:17.464422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52769f8-7bf4-4a06-a2a0-d7433ecb93ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:17.465351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52769f8-7bf4-4a06-a2a0-d7433ecb93ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:17.465794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52769f8-7bf4-4a06-a2a0-d7433ecb93ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:17.466113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52769f8-7bf4-4a06-a2a0-d7433ecb93ff admin admin] PATH_INFO: `/v3/projects/839e3b89bf2f4a8da5c27e2255bee14e/users/b0cc6ef3dd94428e95815e330db38288/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:17.504482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52769f8-7bf4-4a06-a2a0-d7433ecb93ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=839e3b89bf2f4a8da5c27e2255bee14e, user_id=b0cc6ef3dd94428e95815e330db38288, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:17.507243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52769f8-7bf4-4a06-a2a0-d7433ecb93ff admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:17.530287 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1002/1956] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:17 2022] PUT /identity/v3/projects/839e3b89bf2f4a8da5c27e2255bee14e/users/b0cc6ef3dd94428e95815e330db38288/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:17.540766 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121cd15-eb00-477b-baa0-029568076109 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:17.541050 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121cd15-eb00-477b-baa0-029568076109 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:17.541237 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121cd15-eb00-477b-baa0-029568076109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:17.574494 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-6121cd15-eb00-477b-baa0-029568076109 None None] MFA Rules not processed for user `b0cc6ef3dd94428e95815e330db38288`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:17.576009 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6121cd15-eb00-477b-baa0-029568076109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:17.706370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 955/1957] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:17.711102 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52769f8-7bf4-4a06-a2a0-d7433ecb93ff admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:17.839392 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52769f8-7bf4-4a06-a2a0-d7433ecb93ff admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:17.857425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47621e5a-896c-4efe-8e30-491d3259d2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:17.858156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47621e5a-896c-4efe-8e30-491d3259d2e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:17.858448 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47621e5a-896c-4efe-8e30-491d3259d2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:17.858732 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47621e5a-896c-4efe-8e30-491d3259d2e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:17.859446 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47621e5a-896c-4efe-8e30-491d3259d2e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:17.861459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47621e5a-896c-4efe-8e30-491d3259d2e8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:17.914478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1003/1958] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:17 2022] POST /identity/v3/projects => generated 367 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:17.915926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121cd15-eb00-477b-baa0-029568076109 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:17.930604 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121cd15-eb00-477b-baa0-029568076109 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:18.022728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47621e5a-896c-4efe-8e30-491d3259d2e8 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:18.025901 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-47621e5a-896c-4efe-8e30-491d3259d2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:18.078118 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe95126-7515-423b-a398-844d38deb780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:18.085293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe95126-7515-423b-a398-844d38deb780 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:18.086532 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe95126-7515-423b-a398-844d38deb780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:18.087898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe95126-7515-423b-a398-844d38deb780 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:18.090597 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe95126-7515-423b-a398-844d38deb780 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:18.095706 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe95126-7515-423b-a398-844d38deb780 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:18.150946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 956/1959] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:17 2022] POST /identity/v3/users => generated 290 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:18.153231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe95126-7515-423b-a398-844d38deb780 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:18.179704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe95126-7515-423b-a398-844d38deb780 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:18.193291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea74e7c-4159-4c91-95fd-1dee00f7a30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:18.194210 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea74e7c-4159-4c91-95fd-1dee00f7a30a admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:18.195585 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea74e7c-4159-4c91-95fd-1dee00f7a30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:18.195585 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea74e7c-4159-4c91-95fd-1dee00f7a30a admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:18.196090 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47621e5a-896c-4efe-8e30-491d3259d2e8 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:18.196618 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea74e7c-4159-4c91-95fd-1dee00f7a30a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:18.198827 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea74e7c-4159-4c91-95fd-1dee00f7a30a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:18.206890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d41109-f924-45ad-8aa8-e6fa332f3a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:18.211036 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d41109-f924-45ad-8aa8-e6fa332f3a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:18.211530 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d41109-f924-45ad-8aa8-e6fa332f3a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:18.211818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d41109-f924-45ad-8aa8-e6fa332f3a55 admin admin] PATH_INFO: `/v3/users/8039d9c7fca546ebb3ca46333cba968a` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:18.213485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d41109-f924-45ad-8aa8-e6fa332f3a55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8039d9c7fca546ebb3ca46333cba968a)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:18.215947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d41109-f924-45ad-8aa8-e6fa332f3a55 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:18.228157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 957/1960] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:18.232051 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea74e7c-4159-4c91-95fd-1dee00f7a30a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:18.234388 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:18.262033 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea74e7c-4159-4c91-95fd-1dee00f7a30a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:18.269777 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41632f38-c82d-43e8-b0b8-36a8321ffec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:18.284544 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41632f38-c82d-43e8-b0b8-36a8321ffec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:18.284966 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41632f38-c82d-43e8-b0b8-36a8321ffec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:18.285254 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41632f38-c82d-43e8-b0b8-36a8321ffec7 admin admin] PATH_INFO: `/v3/projects/c4c17f3df0d545d5a2af51c3cafd9ac5/users/49e442cf479745c4869edfb47374a8b6/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:18.299141 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c0d41109-f924-45ad-8aa8-e6fa332f3a55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8039d9c7fca546ebb3ca46333cba968a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:18.300438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c0d41109-f924-45ad-8aa8-e6fa332f3a55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8039d9c7fca546ebb3ca46333cba968a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:18.301524 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c0d41109-f924-45ad-8aa8-e6fa332f3a55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8039d9c7fca546ebb3ca46333cba968a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:18.327208 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41632f38-c82d-43e8-b0b8-36a8321ffec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4c17f3df0d545d5a2af51c3cafd9ac5, user_id=49e442cf479745c4869edfb47374a8b6, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:18.335132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c0d41109-f924-45ad-8aa8-e6fa332f3a55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8039d9c7fca546ebb3ca46333cba968a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:18.336184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41632f38-c82d-43e8-b0b8-36a8321ffec7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:18.349394 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c0d41109-f924-45ad-8aa8-e6fa332f3a55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8039d9c7fca546ebb3ca46333cba968a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:18.363592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 958/1961] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:18 2022] PUT /identity/v3/projects/c4c17f3df0d545d5a2af51c3cafd9ac5/users/49e442cf479745c4869edfb47374a8b6/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:43:18.368106 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41632f38-c82d-43e8-b0b8-36a8321ffec7 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:18.370292 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-41632f38-c82d-43e8-b0b8-36a8321ffec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:18.381476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1004/1962] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:18 2022] DELETE /identity/v3/users/8039d9c7fca546ebb3ca46333cba968a => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:18.386957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adcddf-be13-4317-be71-cde3cbb30ab5 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:18.387843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adcddf-be13-4317-be71-cde3cbb30ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:18.388825 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adcddf-be13-4317-be71-cde3cbb30ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:18.425207 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-79adcddf-be13-4317-be71-cde3cbb30ab5 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:18.426893 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-79adcddf-be13-4317-be71-cde3cbb30ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:18.467274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:18.502391 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41632f38-c82d-43e8-b0b8-36a8321ffec7 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:18.557096 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1005/1963] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:43:18.628912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cfdb1-f954-4fc6-989f-a25f2a6c8130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:18.630186 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cfdb1-f954-4fc6-989f-a25f2a6c8130 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:18.630441 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cfdb1-f954-4fc6-989f-a25f2a6c8130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:18.630710 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cfdb1-f954-4fc6-989f-a25f2a6c8130 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:18.631593 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0cfdb1-f954-4fc6-989f-a25f2a6c8130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:18.639550 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0cfdb1-f954-4fc6-989f-a25f2a6c8130 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:18.642775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adcddf-be13-4317-be71-cde3cbb30ab5 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:18.644752 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-79adcddf-be13-4317-be71-cde3cbb30ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:18.693602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 959/1964] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:18.694925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cfdb1-f954-4fc6-989f-a25f2a6c8130 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:18.707999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cfdb1-f954-4fc6-989f-a25f2a6c8130 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:18.721802 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e303a89-1aed-4291-85a5-e17ed4f89105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:18.722807 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e303a89-1aed-4291-85a5-e17ed4f89105 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:18.723089 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e303a89-1aed-4291-85a5-e17ed4f89105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:18.723687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e303a89-1aed-4291-85a5-e17ed4f89105 admin admin] PATH_INFO: `/v3/projects/c4c17f3df0d545d5a2af51c3cafd9ac5/users/49e442cf479745c4869edfb47374a8b6/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:18.726220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e303a89-1aed-4291-85a5-e17ed4f89105 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4c17f3df0d545d5a2af51c3cafd9ac5, user_id=49e442cf479745c4869edfb47374a8b6, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:18.729505 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e303a89-1aed-4291-85a5-e17ed4f89105 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:18.749712 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 960/1965] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:18 2022] PUT /identity/v3/projects/c4c17f3df0d545d5a2af51c3cafd9ac5/users/49e442cf479745c4869edfb47374a8b6/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:18.764584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e2c0e-3335-4bdd-b6aa-dee22a66f42a None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:18.764584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e2c0e-3335-4bdd-b6aa-dee22a66f42a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:18.764584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e2c0e-3335-4bdd-b6aa-dee22a66f42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:18.793903 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adcddf-be13-4317-be71-cde3cbb30ab5 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:18.821816 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-248e2c0e-3335-4bdd-b6aa-dee22a66f42a None None] MFA Rules not processed for user `49e442cf479745c4869edfb47374a8b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:18.824351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-248e2c0e-3335-4bdd-b6aa-dee22a66f42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:18.916623 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:18.927928 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a91cc9-524c-4b76-928e-a20b61c8195e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:18.929166 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a91cc9-524c-4b76-928e-a20b61c8195e service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:18.929529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a91cc9-524c-4b76-928e-a20b61c8195e service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:18.930827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a91cc9-524c-4b76-928e-a20b61c8195e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:18.991689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 961/1966] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:18 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:19.001768 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e2c0e-3335-4bdd-b6aa-dee22a66f42a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:19.003476 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-248e2c0e-3335-4bdd-b6aa-dee22a66f42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:19.072829 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a91cc9-524c-4b76-928e-a20b61c8195e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:19.075273 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a91cc9-524c-4b76-928e-a20b61c8195e service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:19.089358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1006/1967] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:19.123420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e2c0e-3335-4bdd-b6aa-dee22a66f42a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:19.128557 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde11c02-8e32-4c55-94b3-3d4be484e5fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:19.129227 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde11c02-8e32-4c55-94b3-3d4be484e5fa service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:19.131369 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde11c02-8e32-4c55-94b3-3d4be484e5fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:19.131655 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde11c02-8e32-4c55-94b3-3d4be484e5fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:19.156112 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde11c02-8e32-4c55-94b3-3d4be484e5fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:19.157592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde11c02-8e32-4c55-94b3-3d4be484e5fa service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:19.172615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:19.194168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 962/1968] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:19.352019 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a91cc9-524c-4b76-928e-a20b61c8195e service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:19.374446 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a91cc9-524c-4b76-928e-a20b61c8195e service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:19.386821 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad443aae-42d8-4081-8297-904fe7dbe4dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:19.387495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad443aae-42d8-4081-8297-904fe7dbe4dd service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:19.387779 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad443aae-42d8-4081-8297-904fe7dbe4dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:19.388051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad443aae-42d8-4081-8297-904fe7dbe4dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:19.393687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ad443aae-42d8-4081-8297-904fe7dbe4dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:19.582309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad443aae-42d8-4081-8297-904fe7dbe4dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:19.586909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad443aae-42d8-4081-8297-904fe7dbe4dd service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:19.604984 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1007/1969] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:19 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:20.131154 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde11c02-8e32-4c55-94b3-3d4be484e5fa service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:20.132728 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fde11c02-8e32-4c55-94b3-3d4be484e5fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:20.176840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde11c02-8e32-4c55-94b3-3d4be484e5fa service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:20.190014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298755b3-ccbe-4b57-8589-17d18fe79319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:20.192795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298755b3-ccbe-4b57-8589-17d18fe79319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:20.199380 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298755b3-ccbe-4b57-8589-17d18fe79319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:20.200013 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298755b3-ccbe-4b57-8589-17d18fe79319 admin admin] PATH_INFO: `/v3/projects/aff66bc3bdb24fd4be85e282fcb480ea` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:20.201637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298755b3-ccbe-4b57-8589-17d18fe79319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aff66bc3bdb24fd4be85e282fcb480ea)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:20.204183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298755b3-ccbe-4b57-8589-17d18fe79319 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:20.298117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-298755b3-ccbe-4b57-8589-17d18fe79319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aff66bc3bdb24fd4be85e282fcb480ea', 'request_id': 'req-298755b3-ccbe-4b57-8589-17d18fe79319'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:20.308373 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-298755b3-ccbe-4b57-8589-17d18fe79319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aff66bc3bdb24fd4be85e282fcb480ea', 'request_id': 'req-298755b3-ccbe-4b57-8589-17d18fe79319'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:20.324611 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-298755b3-ccbe-4b57-8589-17d18fe79319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aff66bc3bdb24fd4be85e282fcb480ea', 'request_id': 'req-298755b3-ccbe-4b57-8589-17d18fe79319'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:20.346744 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-298755b3-ccbe-4b57-8589-17d18fe79319 admin admin] The token cache is being invalidate because project aff66bc3bdb24fd4be85e282fcb480ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:20.347221 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-298755b3-ccbe-4b57-8589-17d18fe79319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:20.348045 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-298755b3-ccbe-4b57-8589-17d18fe79319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:20.350775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 963/1970] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:20 2022] DELETE /identity/v3/projects/aff66bc3bdb24fd4be85e282fcb480ea => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:20.355595 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24724c89-b5f3-4b4d-8e32-67a6719f52b7 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:20.356430 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24724c89-b5f3-4b4d-8e32-67a6719f52b7 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:20.357568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24724c89-b5f3-4b4d-8e32-67a6719f52b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:20.402496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-24724c89-b5f3-4b4d-8e32-67a6719f52b7 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:20.407208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-24724c89-b5f3-4b4d-8e32-67a6719f52b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:20.527419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1008/1971] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:20.537212 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59473b25-1661-4912-8dfb-aadbe6966850 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:20.537729 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59473b25-1661-4912-8dfb-aadbe6966850 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:20.538267 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59473b25-1661-4912-8dfb-aadbe6966850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:20.577368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-59473b25-1661-4912-8dfb-aadbe6966850 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:20.578853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-59473b25-1661-4912-8dfb-aadbe6966850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:20.619428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 964/1972] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:20.623647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24724c89-b5f3-4b4d-8e32-67a6719f52b7 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:20.638311 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24724c89-b5f3-4b4d-8e32-67a6719f52b7 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:20.659117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41994133-c8b5-4aa4-84ef-4c23767ec429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:20.661626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41994133-c8b5-4aa4-84ef-4c23767ec429 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:20.662369 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41994133-c8b5-4aa4-84ef-4c23767ec429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:20.665876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41994133-c8b5-4aa4-84ef-4c23767ec429 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:20.666860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41994133-c8b5-4aa4-84ef-4c23767ec429 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:43:20.667280 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41994133-c8b5-4aa4-84ef-4c23767ec429 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:20.669860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41994133-c8b5-4aa4-84ef-4c23767ec429 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:20.686160 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1009/1973] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:43:20.691689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59473b25-1661-4912-8dfb-aadbe6966850 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:20.708519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59473b25-1661-4912-8dfb-aadbe6966850 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:20.714870 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41ec2f-13d5-498b-81c7-eb17ad1ddcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:20.715727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41ec2f-13d5-498b-81c7-eb17ad1ddcda admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:20.716230 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41ec2f-13d5-498b-81c7-eb17ad1ddcda admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:20.716648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41ec2f-13d5-498b-81c7-eb17ad1ddcda admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:20.717541 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41ec2f-13d5-498b-81c7-eb17ad1ddcda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:20.719605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41ec2f-13d5-498b-81c7-eb17ad1ddcda admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:20.757506 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573ab0d-85b6-4fc6-a18e-78cd51e297e7 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:20.757810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573ab0d-85b6-4fc6-a18e-78cd51e297e7 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:20.758256 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573ab0d-85b6-4fc6-a18e-78cd51e297e7 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:20.759959 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1010/1974] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jul 12 13:43:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:43:20.768261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 965/1975] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:20 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:43:20.769894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573ab0d-85b6-4fc6-a18e-78cd51e297e7 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:20.782235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa58ba-9af9-40e0-8f6c-2a024038f785 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:20.782552 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa58ba-9af9-40e0-8f6c-2a024038f785 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:20.782811 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa58ba-9af9-40e0-8f6c-2a024038f785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:20.834897 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:20.837355 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e9aa58ba-9af9-40e0-8f6c-2a024038f785 None None] MFA Rules not processed for user `f1308deea1ea4629a1ac22d4b903ad4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:20.839406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e9aa58ba-9af9-40e0-8f6c-2a024038f785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:20.883213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573ab0d-85b6-4fc6-a18e-78cd51e297e7 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:20.888239 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c15722-c8f4-4f40-b1e4-74cf76a13344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:20.888896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c15722-c8f4-4f40-b1e4-74cf76a13344 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:20.889501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c15722-c8f4-4f40-b1e4-74cf76a13344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:20.889879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c15722-c8f4-4f40-b1e4-74cf76a13344 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:20.891029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c15722-c8f4-4f40-b1e4-74cf76a13344 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:20.893112 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c15722-c8f4-4f40-b1e4-74cf76a13344 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:20.921040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1011/1976] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:20 2022] POST /identity/v3/users => generated 287 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:20.922410 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c15722-c8f4-4f40-b1e4-74cf76a13344 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:20.924469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b6c15722-c8f4-4f40-b1e4-74cf76a13344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:20.938883 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c15722-c8f4-4f40-b1e4-74cf76a13344 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:20.945213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd66a1b-dfb9-4a6b-9f80-05b6a4b6c820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:20.946145 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd66a1b-dfb9-4a6b-9f80-05b6a4b6c820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:20.946595 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd66a1b-dfb9-4a6b-9f80-05b6a4b6c820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:20.946884 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd66a1b-dfb9-4a6b-9f80-05b6a4b6c820 admin admin] PATH_INFO: `/v3/users/c25bdc429c3c4f8ba33ee4042f128c8e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:20.948403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd66a1b-dfb9-4a6b-9f80-05b6a4b6c820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c25bdc429c3c4f8ba33ee4042f128c8e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:20.950412 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd66a1b-dfb9-4a6b-9f80-05b6a4b6c820 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:20.969629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 966/1977] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jul 12 13:43:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:20.971056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa58ba-9af9-40e0-8f6c-2a024038f785 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:20.984641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa58ba-9af9-40e0-8f6c-2a024038f785 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:20.993393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70300f-0ec5-4d16-b370-5ba6b4e3e639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:20.994688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70300f-0ec5-4d16-b370-5ba6b4e3e639 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:20.995083 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70300f-0ec5-4d16-b370-5ba6b4e3e639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:20.995304 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70300f-0ec5-4d16-b370-5ba6b4e3e639 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:20.996301 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70300f-0ec5-4d16-b370-5ba6b4e3e639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:20.998584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70300f-0ec5-4d16-b370-5ba6b4e3e639 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:21.024960 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 967/1978] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:21.025661 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70300f-0ec5-4d16-b370-5ba6b4e3e639 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:21.032837 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4c70300f-0ec5-4d16-b370-5ba6b4e3e639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:21.036885 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ecd66a1b-dfb9-4a6b-9f80-05b6a4b6c820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c25bdc429c3c4f8ba33ee4042f128c8e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:21.038113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ecd66a1b-dfb9-4a6b-9f80-05b6a4b6c820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c25bdc429c3c4f8ba33ee4042f128c8e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:21.039455 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ecd66a1b-dfb9-4a6b-9f80-05b6a4b6c820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c25bdc429c3c4f8ba33ee4042f128c8e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:21.066995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70300f-0ec5-4d16-b370-5ba6b4e3e639 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:21.069890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ecd66a1b-dfb9-4a6b-9f80-05b6a4b6c820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c25bdc429c3c4f8ba33ee4042f128c8e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:21.082277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc9fc8-3967-41b4-8416-c9af6fd1c22c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1308deea1ea4629a1ac22d4b903ad4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:21.084310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc9fc8-3967-41b4-8416-c9af6fd1c22c service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:21.084597 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc9fc8-3967-41b4-8416-c9af6fd1c22c service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:21.085336 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc9fc8-3967-41b4-8416-c9af6fd1c22c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:21.088879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ecd66a1b-dfb9-4a6b-9f80-05b6a4b6c820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c25bdc429c3c4f8ba33ee4042f128c8e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:21.091812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-36fc9fc8-3967-41b4-8416-c9af6fd1c22c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:21.117308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1012/1979] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:20 2022] DELETE /identity/v3/users/c25bdc429c3c4f8ba33ee4042f128c8e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:21.125859 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd66a1b-dfb9-4a6b-9f80-05b6a4b6c820 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:21.129706 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc9fc8-3967-41b4-8416-c9af6fd1c22c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:21.131982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ecd66a1b-dfb9-4a6b-9f80-05b6a4b6c820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:21.133546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc9fc8-3967-41b4-8416-c9af6fd1c22c service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:21.151395 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 968/1980] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:21.249887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd66a1b-dfb9-4a6b-9f80-05b6a4b6c820 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:21.265229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb4abf-5081-42b5-85f6-71b51c5c1553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:21.266622 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb4abf-5081-42b5-85f6-71b51c5c1553 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:21.267317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb4abf-5081-42b5-85f6-71b51c5c1553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:21.267868 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb4abf-5081-42b5-85f6-71b51c5c1553 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:21.269492 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb4abf-5081-42b5-85f6-71b51c5c1553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:21.275745 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb4abf-5081-42b5-85f6-71b51c5c1553 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:21.305870 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1013/1981] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:21.311546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc9fc8-3967-41b4-8416-c9af6fd1c22c service placement] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:21.344181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc9fc8-3967-41b4-8416-c9af6fd1c22c service placement] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:21.350829 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317c0a4-fc63-46e8-8e05-3864b8691d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:21.352196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317c0a4-fc63-46e8-8e05-3864b8691d42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:21.352482 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317c0a4-fc63-46e8-8e05-3864b8691d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:21.353064 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317c0a4-fc63-46e8-8e05-3864b8691d42 admin admin] PATH_INFO: `/v3/projects/2307f2bbc9c9490f899a06ca76633a96/users/254417a47d4242f5ab60c72e1d9df70f/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:21.383975 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317c0a4-fc63-46e8-8e05-3864b8691d42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2307f2bbc9c9490f899a06ca76633a96, user_id=254417a47d4242f5ab60c72e1d9df70f, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:21.390510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317c0a4-fc63-46e8-8e05-3864b8691d42 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:21.422620 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 969/1982] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:21 2022] PUT /identity/v3/projects/2307f2bbc9c9490f899a06ca76633a96/users/254417a47d4242f5ab60c72e1d9df70f/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:21.433086 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482718d-0d9f-4fce-ac82-722f1b22e6b7 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:21.433632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482718d-0d9f-4fce-ac82-722f1b22e6b7 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:21.433738 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482718d-0d9f-4fce-ac82-722f1b22e6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:21.471764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-0482718d-0d9f-4fce-ac82-722f1b22e6b7 None None] MFA Rules not processed for user `254417a47d4242f5ab60c72e1d9df70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:21.473000 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0482718d-0d9f-4fce-ac82-722f1b22e6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:21.485877 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:21.545124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317c0a4-fc63-46e8-8e05-3864b8691d42 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:21.546941 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6317c0a4-fc63-46e8-8e05-3864b8691d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:21.553498 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:21.605612 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1014/1983] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:21.623051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482718d-0d9f-4fce-ac82-722f1b22e6b7 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:21.625143 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0482718d-0d9f-4fce-ac82-722f1b22e6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:21.720046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317c0a4-fc63-46e8-8e05-3864b8691d42 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:21.727109 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88046c-2963-4e7c-a8c1-da939a45fc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:21.728054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88046c-2963-4e7c-a8c1-da939a45fc2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:21.728498 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88046c-2963-4e7c-a8c1-da939a45fc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:21.728788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88046c-2963-4e7c-a8c1-da939a45fc2f admin admin] PATH_INFO: `/v3/projects/3f5464e5ca174b398ebb28ad3e8a84f1` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:21.731755 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88046c-2963-4e7c-a8c1-da939a45fc2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f5464e5ca174b398ebb28ad3e8a84f1)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:21.733726 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88046c-2963-4e7c-a8c1-da939a45fc2f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:21.758726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482718d-0d9f-4fce-ac82-722f1b22e6b7 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:21.775916 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228dc64-5b20-44b8-a5d5-1a52fe13e96c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:21.778727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228dc64-5b20-44b8-a5d5-1a52fe13e96c service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:21.779193 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228dc64-5b20-44b8-a5d5-1a52fe13e96c service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:21.779624 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228dc64-5b20-44b8-a5d5-1a52fe13e96c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:21.802281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228dc64-5b20-44b8-a5d5-1a52fe13e96c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:21.803924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228dc64-5b20-44b8-a5d5-1a52fe13e96c service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:21.818990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1015/1984] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:21.835724 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8e88046c-2963-4e7c-a8c1-da939a45fc2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f5464e5ca174b398ebb28ad3e8a84f1', 'request_id': 'req-8e88046c-2963-4e7c-a8c1-da939a45fc2f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:21.850196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8e88046c-2963-4e7c-a8c1-da939a45fc2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f5464e5ca174b398ebb28ad3e8a84f1', 'request_id': 'req-8e88046c-2963-4e7c-a8c1-da939a45fc2f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:21.867202 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8e88046c-2963-4e7c-a8c1-da939a45fc2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f5464e5ca174b398ebb28ad3e8a84f1', 'request_id': 'req-8e88046c-2963-4e7c-a8c1-da939a45fc2f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:21.892368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8e88046c-2963-4e7c-a8c1-da939a45fc2f admin admin] The token cache is being invalidate because project 3f5464e5ca174b398ebb28ad3e8a84f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:21.892666 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8e88046c-2963-4e7c-a8c1-da939a45fc2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:21.894078 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8e88046c-2963-4e7c-a8c1-da939a45fc2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:21.896990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 970/1985] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:21 2022] DELETE /identity/v3/projects/3f5464e5ca174b398ebb28ad3e8a84f1 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:21.902127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cc036-4ed8-48b8-b14d-c338badd879a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:21.903062 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cc036-4ed8-48b8-b14d-c338badd879a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:21.903423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cc036-4ed8-48b8-b14d-c338badd879a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:21.939184 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ee5cc036-4ed8-48b8-b14d-c338badd879a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:21.940879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ee5cc036-4ed8-48b8-b14d-c338badd879a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:22.060360 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1016/1986] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:22.068509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46475f8d-907e-46a7-ad42-1200a593c7d2 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:22.068800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46475f8d-907e-46a7-ad42-1200a593c7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:22.069080 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46475f8d-907e-46a7-ad42-1200a593c7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:22.102885 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-46475f8d-907e-46a7-ad42-1200a593c7d2 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:22.104164 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-46475f8d-907e-46a7-ad42-1200a593c7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:22.126229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 971/1987] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:22.130955 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cc036-4ed8-48b8-b14d-c338badd879a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:22.144621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cc036-4ed8-48b8-b14d-c338badd879a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:22.152019 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c965c9e-90ec-46e5-8550-5c06920ee609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:22.152991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c965c9e-90ec-46e5-8550-5c06920ee609 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:22.153390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c965c9e-90ec-46e5-8550-5c06920ee609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:22.153776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c965c9e-90ec-46e5-8550-5c06920ee609 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:22.154671 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c965c9e-90ec-46e5-8550-5c06920ee609 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:43:22.169741 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c965c9e-90ec-46e5-8550-5c06920ee609 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:22.171983 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c965c9e-90ec-46e5-8550-5c06920ee609 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:22.213878 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1017/1988] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:43:22.218088 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46475f8d-907e-46a7-ad42-1200a593c7d2 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:22.236363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46475f8d-907e-46a7-ad42-1200a593c7d2 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:22.260326 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21160e-39fe-45f2-b1eb-cdd1e3018e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:22.261598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21160e-39fe-45f2-b1eb-cdd1e3018e68 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:22.261891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21160e-39fe-45f2-b1eb-cdd1e3018e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:22.262213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21160e-39fe-45f2-b1eb-cdd1e3018e68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:22.262969 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf21160e-39fe-45f2-b1eb-cdd1e3018e68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:22.264916 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf21160e-39fe-45f2-b1eb-cdd1e3018e68 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:22.309424 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 972/1989] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:22 2022] POST /identity/v3/projects => generated 369 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:22.311308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c965c9e-90ec-46e5-8550-5c06920ee609 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:22.333057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c965c9e-90ec-46e5-8550-5c06920ee609 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:22.386773 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:22.408046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a4dd1-4909-4249-a8bd-f6c9c5ca5897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:22.408801 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a4dd1-4909-4249-a8bd-f6c9c5ca5897 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:22.409091 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a4dd1-4909-4249-a8bd-f6c9c5ca5897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:22.409797 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a4dd1-4909-4249-a8bd-f6c9c5ca5897 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:22.410177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a4dd1-4909-4249-a8bd-f6c9c5ca5897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:22.413056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a4dd1-4909-4249-a8bd-f6c9c5ca5897 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:22.451215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1018/1990] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:22 2022] POST /identity/v3/users => generated 285 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:22.466882 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21160e-39fe-45f2-b1eb-cdd1e3018e68 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:22.496227 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21160e-39fe-45f2-b1eb-cdd1e3018e68 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:22.503586 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aba0ba-4686-49fc-b780-07c03d6a068b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:22.504571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aba0ba-4686-49fc-b780-07c03d6a068b admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:22.504920 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aba0ba-4686-49fc-b780-07c03d6a068b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:22.505259 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aba0ba-4686-49fc-b780-07c03d6a068b admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:22.506614 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aba0ba-4686-49fc-b780-07c03d6a068b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:22.508938 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aba0ba-4686-49fc-b780-07c03d6a068b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:22.532995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 973/1991] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:22.537741 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a4dd1-4909-4249-a8bd-f6c9c5ca5897 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:22.549061 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a4dd1-4909-4249-a8bd-f6c9c5ca5897 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:22.554669 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672af70-bff5-4d91-9c05-f80508f7623b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:22.555595 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672af70-bff5-4d91-9c05-f80508f7623b admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:22.556037 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672af70-bff5-4d91-9c05-f80508f7623b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:22.556467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672af70-bff5-4d91-9c05-f80508f7623b admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:22.557658 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672af70-bff5-4d91-9c05-f80508f7623b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:22.559512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672af70-bff5-4d91-9c05-f80508f7623b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:22.574451 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1019/1992] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:22.579589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aba0ba-4686-49fc-b780-07c03d6a068b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:22.602123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc36428-89a8-4acf-a5b4-a84f9ec961b7 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:22.602685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc36428-89a8-4acf-a5b4-a84f9ec961b7 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:22.602923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aba0ba-4686-49fc-b780-07c03d6a068b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:22.603484 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc36428-89a8-4acf-a5b4-a84f9ec961b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:22.613926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d801d5d-4692-4a74-bc0a-ebcd760c7373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:22.615063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d801d5d-4692-4a74-bc0a-ebcd760c7373 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:22.616270 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d801d5d-4692-4a74-bc0a-ebcd760c7373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:22.616583 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d801d5d-4692-4a74-bc0a-ebcd760c7373 admin admin] PATH_INFO: `/v3/projects/9270edf0542146ba81bbc81ca8a3df53/users/041c58e80a404f3ab75ab4fc4afafbdd/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:22.637201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d801d5d-4692-4a74-bc0a-ebcd760c7373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9270edf0542146ba81bbc81ca8a3df53, user_id=041c58e80a404f3ab75ab4fc4afafbdd, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:22.653653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-bbc36428-89a8-4acf-a5b4-a84f9ec961b7 None None] MFA Rules not processed for user `bb11db914a57439bada24b5f755a143a`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:22.655023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-bbc36428-89a8-4acf-a5b4-a84f9ec961b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:22.656619 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d801d5d-4692-4a74-bc0a-ebcd760c7373 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:22.679984 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 974/1993] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:22 2022] PUT /identity/v3/projects/9270edf0542146ba81bbc81ca8a3df53/users/041c58e80a404f3ab75ab4fc4afafbdd/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:22.691492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c375282-ff89-4373-9c3c-907e0cf339a5 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:22.691958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c375282-ff89-4373-9c3c-907e0cf339a5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:22.692308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c375282-ff89-4373-9c3c-907e0cf339a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:22.724152 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-5c375282-ff89-4373-9c3c-907e0cf339a5 None None] MFA Rules not processed for user `041c58e80a404f3ab75ab4fc4afafbdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:22.725391 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5c375282-ff89-4373-9c3c-907e0cf339a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:22.749382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1020/1994] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:43:22.750909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:22.755431 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc36428-89a8-4acf-a5b4-a84f9ec961b7 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:22.758248 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-bbc36428-89a8-4acf-a5b4-a84f9ec961b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:22.774303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc36428-89a8-4acf-a5b4-a84f9ec961b7 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:22.847975 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 975/1995] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:22 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:22.850003 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c375282-ff89-4373-9c3c-907e0cf339a5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:22.874203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e801e-a595-49df-83ca-b94fc1298180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:22.874970 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e801e-a595-49df-83ca-b94fc1298180 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:22.875483 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e801e-a595-49df-83ca-b94fc1298180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:22.875769 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e801e-a595-49df-83ca-b94fc1298180 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:22.876856 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e801e-a595-49df-83ca-b94fc1298180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:22.879124 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e801e-a595-49df-83ca-b94fc1298180 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:22.920776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1021/1996] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:22 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:22.924850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e801e-a595-49df-83ca-b94fc1298180 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:22.956030 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c375282-ff89-4373-9c3c-907e0cf339a5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:22.973315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:23.045139 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e801e-a595-49df-83ca-b94fc1298180 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:23.052023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e0faf-13fa-48bf-a511-50e845ac4d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:23.052715 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e0faf-13fa-48bf-a511-50e845ac4d8d admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:23.053002 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e0faf-13fa-48bf-a511-50e845ac4d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:23.053382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e0faf-13fa-48bf-a511-50e845ac4d8d admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:23.054012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e0faf-13fa-48bf-a511-50e845ac4d8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:23.058802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e0faf-13fa-48bf-a511-50e845ac4d8d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:23.060612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c028aa0-adc3-4a5e-b7f3-784df1ca8220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:23.061501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c028aa0-adc3-4a5e-b7f3-784df1ca8220 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:23.061924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c028aa0-adc3-4a5e-b7f3-784df1ca8220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:23.062377 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c028aa0-adc3-4a5e-b7f3-784df1ca8220 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:23.063218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c028aa0-adc3-4a5e-b7f3-784df1ca8220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:23.065267 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c028aa0-adc3-4a5e-b7f3-784df1ca8220 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:23.093487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1022/1997] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:22 2022] POST /identity/v3/users => generated 295 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:43:23.094860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e0faf-13fa-48bf-a511-50e845ac4d8d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:23.107114 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e0faf-13fa-48bf-a511-50e845ac4d8d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:23.109161 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 976/1998] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:22 2022] POST /identity/v3/projects => generated 369 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:43:23.113071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c028aa0-adc3-4a5e-b7f3-784df1ca8220 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:23.189823 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:23.208790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c028aa0-adc3-4a5e-b7f3-784df1ca8220 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:23.213461 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45959e88-d534-4070-b2af-5d8a8030dea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:23.214168 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45959e88-d534-4070-b2af-5d8a8030dea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:23.214458 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45959e88-d534-4070-b2af-5d8a8030dea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:23.214738 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45959e88-d534-4070-b2af-5d8a8030dea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:23.214839 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d77be4-3c7b-43ae-a034-c0c4549b4539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:23.215614 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45959e88-d534-4070-b2af-5d8a8030dea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:23.215706 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d77be4-3c7b-43ae-a034-c0c4549b4539 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:23.216196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d77be4-3c7b-43ae-a034-c0c4549b4539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:23.216548 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d77be4-3c7b-43ae-a034-c0c4549b4539 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:23.217135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45959e88-d534-4070-b2af-5d8a8030dea5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:23.217378 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d77be4-3c7b-43ae-a034-c0c4549b4539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:23.220365 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d77be4-3c7b-43ae-a034-c0c4549b4539 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:23.228688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1023/1999] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:23.233329 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45959e88-d534-4070-b2af-5d8a8030dea5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:23.248115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45959e88-d534-4070-b2af-5d8a8030dea5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:23.260264 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a0435-216c-4038-a631-c741704bec85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:23.263665 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 977/2000] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:23 2022] POST /identity/v3/users => generated 291 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:43:23.263764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a0435-216c-4038-a631-c741704bec85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:23.263764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a0435-216c-4038-a631-c741704bec85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:23.263764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a0435-216c-4038-a631-c741704bec85 admin admin] PATH_INFO: `/v3/projects/29f0628d57db4d9c8c6408298f4af853/users/ebc1b31030634b5eb17d3549959f5507/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:23.266813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d77be4-3c7b-43ae-a034-c0c4549b4539 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:23.280091 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a0435-216c-4038-a631-c741704bec85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29f0628d57db4d9c8c6408298f4af853, user_id=ebc1b31030634b5eb17d3549959f5507, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:23.283412 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a0435-216c-4038-a631-c741704bec85 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:23.286617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d77be4-3c7b-43ae-a034-c0c4549b4539 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:23.299921 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09a664-d985-4bc5-86fd-dd20b70f494d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:23.300641 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1024/2001] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:23 2022] PUT /identity/v3/projects/29f0628d57db4d9c8c6408298f4af853/users/ebc1b31030634b5eb17d3549959f5507/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:23.305178 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09a664-d985-4bc5-86fd-dd20b70f494d admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:23.305178 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09a664-d985-4bc5-86fd-dd20b70f494d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:23.305347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09a664-d985-4bc5-86fd-dd20b70f494d admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:23.306819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09a664-d985-4bc5-86fd-dd20b70f494d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:23.309515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a0435-216c-4038-a631-c741704bec85 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:23.309646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09a664-d985-4bc5-86fd-dd20b70f494d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:23.340493 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 978/2002] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:23.345326 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09a664-d985-4bc5-86fd-dd20b70f494d admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:23.372322 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09a664-d985-4bc5-86fd-dd20b70f494d admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:23.455443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a0435-216c-4038-a631-c741704bec85 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:23.474037 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f2122-d304-4178-b462-a1bb3ade5383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:23.475173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f2122-d304-4178-b462-a1bb3ade5383 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:23.475573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f2122-d304-4178-b462-a1bb3ade5383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:23.476025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f2122-d304-4178-b462-a1bb3ade5383 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:23.477242 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f2122-d304-4178-b462-a1bb3ade5383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:23.479211 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f2122-d304-4178-b462-a1bb3ade5383 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:23.491448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aacd90-34ae-4143-a736-c1eafadea5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:23.492362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aacd90-34ae-4143-a736-c1eafadea5b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:23.492647 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aacd90-34ae-4143-a736-c1eafadea5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:23.493035 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aacd90-34ae-4143-a736-c1eafadea5b6 admin admin] PATH_INFO: `/v3/projects/22a2037e8e3640eb8ed09e352d112796/users/097aef38d7a1446a80111e8d7b8e40a7/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:23.493116 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1025/2003] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:23.496880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f2122-d304-4178-b462-a1bb3ade5383 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:23.513783 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f2122-d304-4178-b462-a1bb3ade5383 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:23.519523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418fa7f-2b2b-41e4-8348-a0e7595e7369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:23.520593 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418fa7f-2b2b-41e4-8348-a0e7595e7369 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:23.520869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418fa7f-2b2b-41e4-8348-a0e7595e7369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:23.521196 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418fa7f-2b2b-41e4-8348-a0e7595e7369 admin admin] PATH_INFO: `/v3/projects/29f0628d57db4d9c8c6408298f4af853/users/ebc1b31030634b5eb17d3549959f5507/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:23.523459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418fa7f-2b2b-41e4-8348-a0e7595e7369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29f0628d57db4d9c8c6408298f4af853, user_id=ebc1b31030634b5eb17d3549959f5507, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:23.527374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418fa7f-2b2b-41e4-8348-a0e7595e7369 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:23.538934 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aacd90-34ae-4143-a736-c1eafadea5b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a2037e8e3640eb8ed09e352d112796, user_id=097aef38d7a1446a80111e8d7b8e40a7, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:23.542678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aacd90-34ae-4143-a736-c1eafadea5b6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:23.547346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1026/2004] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:23 2022] PUT /identity/v3/projects/29f0628d57db4d9c8c6408298f4af853/users/ebc1b31030634b5eb17d3549959f5507/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:43:23.582980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 979/2005] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:23 2022] PUT /identity/v3/projects/22a2037e8e3640eb8ed09e352d112796/users/097aef38d7a1446a80111e8d7b8e40a7/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:23.588128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418fa7f-2b2b-41e4-8348-a0e7595e7369 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:23.628051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:23.704273 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418fa7f-2b2b-41e4-8348-a0e7595e7369 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:23.832740 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d84a12-c461-4f68-8404-eb6d13e40245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:23.835823 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d84a12-c461-4f68-8404-eb6d13e40245 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:23.836110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d84a12-c461-4f68-8404-eb6d13e40245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:23.837691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d84a12-c461-4f68-8404-eb6d13e40245 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:23.838553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d84a12-c461-4f68-8404-eb6d13e40245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:23.843521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d84a12-c461-4f68-8404-eb6d13e40245 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:23.882783 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1027/2006] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:23.891524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aacd90-34ae-4143-a736-c1eafadea5b6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:23.908749 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aacd90-34ae-4143-a736-c1eafadea5b6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:23.918586 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65364489-f34c-49fd-9724-2bc2781bcc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:23.919669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65364489-f34c-49fd-9724-2bc2781bcc42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:23.920268 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65364489-f34c-49fd-9724-2bc2781bcc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:23.920553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65364489-f34c-49fd-9724-2bc2781bcc42 admin admin] PATH_INFO: `/v3/projects/22a2037e8e3640eb8ed09e352d112796/users/097aef38d7a1446a80111e8d7b8e40a7/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:23.923802 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65364489-f34c-49fd-9724-2bc2781bcc42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a2037e8e3640eb8ed09e352d112796, user_id=097aef38d7a1446a80111e8d7b8e40a7, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:23.930261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65364489-f34c-49fd-9724-2bc2781bcc42 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:23.952663 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 980/2007] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:23 2022] PUT /identity/v3/projects/22a2037e8e3640eb8ed09e352d112796/users/097aef38d7a1446a80111e8d7b8e40a7/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:23.964488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb4b4a-a853-46e4-aeba-29a423ae526e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:23.964780 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb4b4a-a853-46e4-aeba-29a423ae526e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:23.965061 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb4b4a-a853-46e4-aeba-29a423ae526e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:24.004494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3bfb4b4a-a853-46e4-aeba-29a423ae526e None None] MFA Rules not processed for user `097aef38d7a1446a80111e8d7b8e40a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:24.009836 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3bfb4b4a-a853-46e4-aeba-29a423ae526e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:24.116797 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1028/2008] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:23 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:24.142437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65364489-f34c-49fd-9724-2bc2781bcc42 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:24.144520 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-65364489-f34c-49fd-9724-2bc2781bcc42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:24.224233 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65364489-f34c-49fd-9724-2bc2781bcc42 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:24.228766 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001d709-f47d-487c-9f46-0c9a6971592d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:24.231204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001d709-f47d-487c-9f46-0c9a6971592d service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:24.232381 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001d709-f47d-487c-9f46-0c9a6971592d service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:24.232381 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001d709-f47d-487c-9f46-0c9a6971592d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:24.257260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001d709-f47d-487c-9f46-0c9a6971592d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:24.259100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001d709-f47d-487c-9f46-0c9a6971592d service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:24.272613 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 981/2009] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:24 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:24.350666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb4b4a-a853-46e4-aeba-29a423ae526e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:24.352737 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3bfb4b4a-a853-46e4-aeba-29a423ae526e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:24.369017 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb4b4a-a853-46e4-aeba-29a423ae526e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:24.373318 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5cad6-c122-4424-b73d-7a83179d4efc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:24.374299 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5cad6-c122-4424-b73d-7a83179d4efc service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:24.374720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5cad6-c122-4424-b73d-7a83179d4efc service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:24.374940 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5cad6-c122-4424-b73d-7a83179d4efc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:24.433844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:24.461775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5cad6-c122-4424-b73d-7a83179d4efc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:24.467489 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5cad6-c122-4424-b73d-7a83179d4efc service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:24.480525 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1029/2010] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:24 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:25.536990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001d709-f47d-487c-9f46-0c9a6971592d service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:25.538619 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5001d709-f47d-487c-9f46-0c9a6971592d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:25.618289 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5cad6-c122-4424-b73d-7a83179d4efc service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:25.620698 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-35f5cad6-c122-4424-b73d-7a83179d4efc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:25.669990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:25.715919 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5cad6-c122-4424-b73d-7a83179d4efc service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:25.716168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001d709-f47d-487c-9f46-0c9a6971592d service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:25.727265 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4024a0b-290c-4b91-bf94-368126fd15b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:25.729281 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4024a0b-290c-4b91-bf94-368126fd15b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:25.730427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4024a0b-290c-4b91-bf94-368126fd15b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:25.731157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4024a0b-290c-4b91-bf94-368126fd15b0 admin admin] PATH_INFO: `/v3/users/254417a47d4242f5ab60c72e1d9df70f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:25.733804 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86aad1-fb15-444d-b4a2-8078fd7690f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:25.735230 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4024a0b-290c-4b91-bf94-368126fd15b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=254417a47d4242f5ab60c72e1d9df70f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:25.736170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86aad1-fb15-444d-b4a2-8078fd7690f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:25.739534 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86aad1-fb15-444d-b4a2-8078fd7690f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:25.739939 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86aad1-fb15-444d-b4a2-8078fd7690f8 admin admin] PATH_INFO: `/v3/users/b0cc6ef3dd94428e95815e330db38288` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:25.740536 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4024a0b-290c-4b91-bf94-368126fd15b0 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:25.742887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86aad1-fb15-444d-b4a2-8078fd7690f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0cc6ef3dd94428e95815e330db38288)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:25.748157 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86aad1-fb15-444d-b4a2-8078fd7690f8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:25.809241 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f4024a0b-290c-4b91-bf94-368126fd15b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '254417a47d4242f5ab60c72e1d9df70f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:25.828211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f4024a0b-290c-4b91-bf94-368126fd15b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '254417a47d4242f5ab60c72e1d9df70f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:25.841181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f4024a0b-290c-4b91-bf94-368126fd15b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '254417a47d4242f5ab60c72e1d9df70f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:25.842272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f4024a0b-290c-4b91-bf94-368126fd15b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '254417a47d4242f5ab60c72e1d9df70f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:25.867348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f4024a0b-290c-4b91-bf94-368126fd15b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '254417a47d4242f5ab60c72e1d9df70f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:25.877714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8b86aad1-fb15-444d-b4a2-8078fd7690f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0cc6ef3dd94428e95815e330db38288'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:25.878408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8b86aad1-fb15-444d-b4a2-8078fd7690f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0cc6ef3dd94428e95815e330db38288'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:25.879433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8b86aad1-fb15-444d-b4a2-8078fd7690f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0cc6ef3dd94428e95815e330db38288'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:25.879782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 982/2011] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:25 2022] DELETE /identity/v3/users/254417a47d4242f5ab60c72e1d9df70f => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:25.886137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57771a10-f3dd-47ed-aa76-a52b43a17f4c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:25.886612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57771a10-f3dd-47ed-aa76-a52b43a17f4c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:25.887337 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57771a10-f3dd-47ed-aa76-a52b43a17f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:25.900308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8b86aad1-fb15-444d-b4a2-8078fd7690f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0cc6ef3dd94428e95815e330db38288'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:25.918365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8b86aad1-fb15-444d-b4a2-8078fd7690f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0cc6ef3dd94428e95815e330db38288'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:25.946654 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1030/2012] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:25 2022] DELETE /identity/v3/users/b0cc6ef3dd94428e95815e330db38288 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:25.951634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-57771a10-f3dd-47ed-aa76-a52b43a17f4c None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:25.952902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-57771a10-f3dd-47ed-aa76-a52b43a17f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:25.955851 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e228ac-87fe-433d-976e-e9e5b65c5925 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:25.956469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e228ac-87fe-433d-976e-e9e5b65c5925 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:25.956988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e228ac-87fe-433d-976e-e9e5b65c5925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:26.003289 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d4e228ac-87fe-433d-976e-e9e5b65c5925 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:26.005385 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d4e228ac-87fe-433d-976e-e9e5b65c5925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:26.143320 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1031/2013] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:26.154103 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e228ac-87fe-433d-976e-e9e5b65c5925 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:26.156197 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d4e228ac-87fe-433d-976e-e9e5b65c5925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:26.169771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 983/2014] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:26.170198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:26.181276 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57771a10-f3dd-47ed-aa76-a52b43a17f4c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:26.183138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-57771a10-f3dd-47ed-aa76-a52b43a17f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:26.324069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57771a10-f3dd-47ed-aa76-a52b43a17f4c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:26.326152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e228ac-87fe-433d-976e-e9e5b65c5925 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:26.336863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f777ad-9e79-4d04-834a-56db3846c677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:26.338832 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c8e2b-9090-45f0-a62c-85da8b586935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:26.339711 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c8e2b-9090-45f0-a62c-85da8b586935 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:26.339873 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f777ad-9e79-4d04-834a-56db3846c677 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:26.340173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f777ad-9e79-4d04-834a-56db3846c677 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:26.340461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f777ad-9e79-4d04-834a-56db3846c677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:26.340571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c8e2b-9090-45f0-a62c-85da8b586935 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:26.341030 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c8e2b-9090-45f0-a62c-85da8b586935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:26.364673 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f777ad-9e79-4d04-834a-56db3846c677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:26.366621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f777ad-9e79-4d04-834a-56db3846c677 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:26.368866 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c8e2b-9090-45f0-a62c-85da8b586935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:26.372455 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c8e2b-9090-45f0-a62c-85da8b586935 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:26.395302 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 984/2015] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:26.404797 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1032/2016] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:43:27.306762 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f777ad-9e79-4d04-834a-56db3846c677 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:27.308420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-17f777ad-9e79-4d04-834a-56db3846c677 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:27.329824 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:27.336805 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f777ad-9e79-4d04-834a-56db3846c677 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:27.354362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453fb983-6c40-453f-a29b-df76f27a6ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:27.355800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fb983-6c40-453f-a29b-df76f27a6ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:27.356103 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fb983-6c40-453f-a29b-df76f27a6ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:27.356387 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fb983-6c40-453f-a29b-df76f27a6ceb admin admin] PATH_INFO: `/v3/projects/2307f2bbc9c9490f899a06ca76633a96` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:27.357629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453fb983-6c40-453f-a29b-df76f27a6ceb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2307f2bbc9c9490f899a06ca76633a96)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:27.360044 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453fb983-6c40-453f-a29b-df76f27a6ceb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:27.456933 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-453fb983-6c40-453f-a29b-df76f27a6ceb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2307f2bbc9c9490f899a06ca76633a96', 'request_id': 'req-453fb983-6c40-453f-a29b-df76f27a6ceb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:27.469515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-453fb983-6c40-453f-a29b-df76f27a6ceb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2307f2bbc9c9490f899a06ca76633a96', 'request_id': 'req-453fb983-6c40-453f-a29b-df76f27a6ceb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:27.483864 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-453fb983-6c40-453f-a29b-df76f27a6ceb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2307f2bbc9c9490f899a06ca76633a96', 'request_id': 'req-453fb983-6c40-453f-a29b-df76f27a6ceb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:27.503304 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-453fb983-6c40-453f-a29b-df76f27a6ceb admin admin] The token cache is being invalidate because project 2307f2bbc9c9490f899a06ca76633a96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:27.503600 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-453fb983-6c40-453f-a29b-df76f27a6ceb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:27.504336 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-453fb983-6c40-453f-a29b-df76f27a6ceb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:27.506283 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 985/2017] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:27 2022] DELETE /identity/v3/projects/2307f2bbc9c9490f899a06ca76633a96 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:27.511247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc5a37-36f7-49f8-bd4c-df0396cd742a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:27.511776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc5a37-36f7-49f8-bd4c-df0396cd742a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:27.511866 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc5a37-36f7-49f8-bd4c-df0396cd742a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:27.538724 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7acc5a37-36f7-49f8-bd4c-df0396cd742a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:27.539844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7acc5a37-36f7-49f8-bd4c-df0396cd742a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:27.621677 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1033/2018] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:27.629828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c3d56-ce05-46ec-b4d2-a69a3c647b55 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:27.629828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c3d56-ce05-46ec-b4d2-a69a3c647b55 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:27.630156 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c3d56-ce05-46ec-b4d2-a69a3c647b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:27.657024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-485c3d56-ce05-46ec-b4d2-a69a3c647b55 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:27.658360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-485c3d56-ce05-46ec-b4d2-a69a3c647b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:27.669741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 986/2019] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:27.674077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc5a37-36f7-49f8-bd4c-df0396cd742a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:27.687271 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc5a37-36f7-49f8-bd4c-df0396cd742a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:27.694654 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41586a85-5d7c-4738-9f5f-79715215be46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:27.695547 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41586a85-5d7c-4738-9f5f-79715215be46 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:27.695935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41586a85-5d7c-4738-9f5f-79715215be46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:27.696316 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41586a85-5d7c-4738-9f5f-79715215be46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:27.697386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41586a85-5d7c-4738-9f5f-79715215be46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:43:27.698109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41586a85-5d7c-4738-9f5f-79715215be46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:27.700088 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41586a85-5d7c-4738-9f5f-79715215be46 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:27.717576 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1034/2020] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:43:27.722407 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c3d56-ce05-46ec-b4d2-a69a3c647b55 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:27.739905 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41586a85-5d7c-4738-9f5f-79715215be46 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:27.743090 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c3d56-ce05-46ec-b4d2-a69a3c647b55 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:27.745026 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-41586a85-5d7c-4738-9f5f-79715215be46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:27.774698 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581e5b5-977b-4843-bffe-66bdf32061a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:27.775396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581e5b5-977b-4843-bffe-66bdf32061a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:27.775706 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581e5b5-977b-4843-bffe-66bdf32061a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:27.775989 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581e5b5-977b-4843-bffe-66bdf32061a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:27.776701 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581e5b5-977b-4843-bffe-66bdf32061a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:27.778685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581e5b5-977b-4843-bffe-66bdf32061a5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:27.803103 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41586a85-5d7c-4738-9f5f-79715215be46 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:27.818031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb7313-9292-4923-a258-090d9c583ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:27.818888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb7313-9292-4923-a258-090d9c583ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:27.819183 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb7313-9292-4923-a258-090d9c583ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:27.819471 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb7313-9292-4923-a258-090d9c583ab1 admin admin] PATH_INFO: `/v3/projects/839e3b89bf2f4a8da5c27e2255bee14e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:27.823010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb7313-9292-4923-a258-090d9c583ab1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=839e3b89bf2f4a8da5c27e2255bee14e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:27.828237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb7313-9292-4923-a258-090d9c583ab1 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:27.910743 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 987/2021] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:27 2022] POST /identity/v3/projects => generated 371 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:27.915989 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581e5b5-977b-4843-bffe-66bdf32061a5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:27.937222 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ffeb7313-9292-4923-a258-090d9c583ab1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '839e3b89bf2f4a8da5c27e2255bee14e', 'request_id': 'req-ffeb7313-9292-4923-a258-090d9c583ab1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:27.947981 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ffeb7313-9292-4923-a258-090d9c583ab1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '839e3b89bf2f4a8da5c27e2255bee14e', 'request_id': 'req-ffeb7313-9292-4923-a258-090d9c583ab1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:27.957159 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ffeb7313-9292-4923-a258-090d9c583ab1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '839e3b89bf2f4a8da5c27e2255bee14e', 'request_id': 'req-ffeb7313-9292-4923-a258-090d9c583ab1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:27.972559 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ffeb7313-9292-4923-a258-090d9c583ab1 admin admin] The token cache is being invalidate because project 839e3b89bf2f4a8da5c27e2255bee14e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:27.972559 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ffeb7313-9292-4923-a258-090d9c583ab1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:27.973743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ffeb7313-9292-4923-a258-090d9c583ab1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:27.975833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1035/2022] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:27 2022] DELETE /identity/v3/projects/839e3b89bf2f4a8da5c27e2255bee14e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:27.981064 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb7313-9292-4923-a258-090d9c583ab1 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:27.982789 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ffeb7313-9292-4923-a258-090d9c583ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:28.064752 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:28.132176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581e5b5-977b-4843-bffe-66bdf32061a5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:28.174403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb7313-9292-4923-a258-090d9c583ab1 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:28.184169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a8828-4c65-413f-802e-36802dd19f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:28.187236 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a8828-4c65-413f-802e-36802dd19f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:28.189056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a8828-4c65-413f-802e-36802dd19f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:28.189747 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a8828-4c65-413f-802e-36802dd19f26 admin admin] PATH_INFO: `/v3/users/49e442cf479745c4869edfb47374a8b6` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:28.202872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a8828-4c65-413f-802e-36802dd19f26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49e442cf479745c4869edfb47374a8b6)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:28.213501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a8828-4c65-413f-802e-36802dd19f26 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:28.351433 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff78c6-5719-40c2-9c28-b495d4770bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:28.352188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff78c6-5719-40c2-9c28-b495d4770bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:28.352471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff78c6-5719-40c2-9c28-b495d4770bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:28.352871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff78c6-5719-40c2-9c28-b495d4770bd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:28.353617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff78c6-5719-40c2-9c28-b495d4770bd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:28.359082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff78c6-5719-40c2-9c28-b495d4770bd8 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:28.378680 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:28.416579 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 988/2023] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:27 2022] POST /identity/v3/users => generated 286 bytes in 505 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:28.425505 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5b7a8828-4c65-413f-802e-36802dd19f26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49e442cf479745c4869edfb47374a8b6'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:28.430726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5b7a8828-4c65-413f-802e-36802dd19f26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49e442cf479745c4869edfb47374a8b6'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:28.432596 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff78c6-5719-40c2-9c28-b495d4770bd8 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:28.432782 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5b7a8828-4c65-413f-802e-36802dd19f26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49e442cf479745c4869edfb47374a8b6'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:28.435235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3bff78c6-5719-40c2-9c28-b495d4770bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:28.473075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff78c6-5719-40c2-9c28-b495d4770bd8 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:28.477415 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5b7a8828-4c65-413f-802e-36802dd19f26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49e442cf479745c4869edfb47374a8b6'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:28.487968 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c9ea7-3bcb-431f-a204-f6029f12a0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:28.489201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c9ea7-3bcb-431f-a204-f6029f12a0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:28.489488 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c9ea7-3bcb-431f-a204-f6029f12a0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:28.494594 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c9ea7-3bcb-431f-a204-f6029f12a0f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:28.495502 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c9ea7-3bcb-431f-a204-f6029f12a0f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:28.497356 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c9ea7-3bcb-431f-a204-f6029f12a0f8 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:28.515870 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5b7a8828-4c65-413f-802e-36802dd19f26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49e442cf479745c4869edfb47374a8b6'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:28.523061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 989/2024] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:28.527001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c9ea7-3bcb-431f-a204-f6029f12a0f8 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:28.542456 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c9ea7-3bcb-431f-a204-f6029f12a0f8 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:28.559672 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1036/2025] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:27 2022] DELETE /identity/v3/users/49e442cf479745c4869edfb47374a8b6 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:28.572158 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca3e9e-5745-4744-be3e-455a2bda9515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:28.584266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca3e9e-5745-4744-be3e-455a2bda9515 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:28.589869 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca3e9e-5745-4744-be3e-455a2bda9515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:28.592514 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca3e9e-5745-4744-be3e-455a2bda9515 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:28.593373 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca3e9e-5745-4744-be3e-455a2bda9515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:28.595094 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca3e9e-5745-4744-be3e-455a2bda9515 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:28.649073 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 990/2026] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:28.656195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a8828-4c65-413f-802e-36802dd19f26 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:28.902937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a8828-4c65-413f-802e-36802dd19f26 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:28.926842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618094c-8d2c-4eac-a113-62673b9230aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:28.928522 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618094c-8d2c-4eac-a113-62673b9230aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:28.928988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618094c-8d2c-4eac-a113-62673b9230aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:28.931920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618094c-8d2c-4eac-a113-62673b9230aa admin admin] PATH_INFO: `/v3/projects/4f3ab1e8d5fd4ac7b34c60a4a3cf9c45/users/287144c6346a43d09cdbae463ca9a961/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:28.942969 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca3e9e-5745-4744-be3e-455a2bda9515 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:28.947001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c9ca3e9e-5745-4744-be3e-455a2bda9515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:28.969200 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618094c-8d2c-4eac-a113-62673b9230aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3ab1e8d5fd4ac7b34c60a4a3cf9c45, user_id=287144c6346a43d09cdbae463ca9a961, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:28.972159 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618094c-8d2c-4eac-a113-62673b9230aa admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:28.975196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca3e9e-5745-4744-be3e-455a2bda9515 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:28.985219 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07342f-da15-493d-a616-720182afb069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:28.988059 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07342f-da15-493d-a616-720182afb069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:28.989158 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07342f-da15-493d-a616-720182afb069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:28.990246 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07342f-da15-493d-a616-720182afb069 admin admin] PATH_INFO: `/v3/users/041c58e80a404f3ab75ab4fc4afafbdd` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:28.995639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07342f-da15-493d-a616-720182afb069 admin admin] RBAC: Authorizing `identity:delete_user(user_id=041c58e80a404f3ab75ab4fc4afafbdd)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:29.000820 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07342f-da15-493d-a616-720182afb069 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:29.009761 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1037/2027] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:28 2022] PUT /identity/v3/projects/4f3ab1e8d5fd4ac7b34c60a4a3cf9c45/users/287144c6346a43d09cdbae463ca9a961/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:29.013080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc5bbe-96bb-42cf-969f-a14178c6a6ce None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:29.013566 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc5bbe-96bb-42cf-969f-a14178c6a6ce None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:29.014059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc5bbe-96bb-42cf-969f-a14178c6a6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:29.043243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-51dc5bbe-96bb-42cf-969f-a14178c6a6ce None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:29.043243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-51dc5bbe-96bb-42cf-969f-a14178c6a6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:29.089418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:29.157144 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8c07342f-da15-493d-a616-720182afb069 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '041c58e80a404f3ab75ab4fc4afafbdd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:29.172713 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8c07342f-da15-493d-a616-720182afb069 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '041c58e80a404f3ab75ab4fc4afafbdd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:29.192783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8c07342f-da15-493d-a616-720182afb069 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '041c58e80a404f3ab75ab4fc4afafbdd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:29.195933 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8c07342f-da15-493d-a616-720182afb069 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '041c58e80a404f3ab75ab4fc4afafbdd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:29.218611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1038/2028] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:29.220450 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8c07342f-da15-493d-a616-720182afb069 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '041c58e80a404f3ab75ab4fc4afafbdd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:29.221681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc5bbe-96bb-42cf-969f-a14178c6a6ce None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:29.226012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-51dc5bbe-96bb-42cf-969f-a14178c6a6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:29.235763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 991/2029] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:28 2022] DELETE /identity/v3/users/041c58e80a404f3ab75ab4fc4afafbdd => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:43:29.251185 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07342f-da15-493d-a616-720182afb069 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:29.253848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8c07342f-da15-493d-a616-720182afb069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:29.266287 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc5bbe-96bb-42cf-969f-a14178c6a6ce None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:29.383271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07342f-da15-493d-a616-720182afb069 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:29.395842 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200542c-1c9f-4793-9b3c-b49b654d538b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:29.396625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200542c-1c9f-4793-9b3c-b49b654d538b service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:29.397010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200542c-1c9f-4793-9b3c-b49b654d538b service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:29.397840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200542c-1c9f-4793-9b3c-b49b654d538b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:29.403310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7200542c-1c9f-4793-9b3c-b49b654d538b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:29.492308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6318d-3372-493b-939f-173f7a5ae2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:29.493739 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6318d-3372-493b-939f-173f7a5ae2af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:29.498209 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6318d-3372-493b-939f-173f7a5ae2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:29.498717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6318d-3372-493b-939f-173f7a5ae2af admin admin] PATH_INFO: `/v3/projects/c4c17f3df0d545d5a2af51c3cafd9ac5` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:29.501724 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6318d-3372-493b-939f-173f7a5ae2af admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4c17f3df0d545d5a2af51c3cafd9ac5)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:29.505824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6318d-3372-493b-939f-173f7a5ae2af admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:29.574232 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:29.584122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7200542c-1c9f-4793-9b3c-b49b654d538b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:29.587439 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7200542c-1c9f-4793-9b3c-b49b654d538b service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:29.619402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 992/2030] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:29.620995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70241a13-d19f-4f1b-b2ea-9864de7093ec None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:29.622561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70241a13-d19f-4f1b-b2ea-9864de7093ec None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:29.622772 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70241a13-d19f-4f1b-b2ea-9864de7093ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:29.644389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-70241a13-d19f-4f1b-b2ea-9864de7093ec None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:29.646313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-70241a13-d19f-4f1b-b2ea-9864de7093ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:29.648435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9ef6318d-3372-493b-939f-173f7a5ae2af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4c17f3df0d545d5a2af51c3cafd9ac5', 'request_id': 'req-9ef6318d-3372-493b-939f-173f7a5ae2af'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:29.665836 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9ef6318d-3372-493b-939f-173f7a5ae2af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4c17f3df0d545d5a2af51c3cafd9ac5', 'request_id': 'req-9ef6318d-3372-493b-939f-173f7a5ae2af'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:29.678720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9ef6318d-3372-493b-939f-173f7a5ae2af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4c17f3df0d545d5a2af51c3cafd9ac5', 'request_id': 'req-9ef6318d-3372-493b-939f-173f7a5ae2af'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:29.693440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9ef6318d-3372-493b-939f-173f7a5ae2af admin admin] The token cache is being invalidate because project c4c17f3df0d545d5a2af51c3cafd9ac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:29.693984 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9ef6318d-3372-493b-939f-173f7a5ae2af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:29.695228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9ef6318d-3372-493b-939f-173f7a5ae2af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:29.700447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1039/2031] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:29 2022] DELETE /identity/v3/projects/c4c17f3df0d545d5a2af51c3cafd9ac5 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:29.702832 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb04e0-4b3e-48aa-9bc6-b2b58a6a0ac7 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:29.704169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb04e0-4b3e-48aa-9bc6-b2b58a6a0ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:29.705582 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb04e0-4b3e-48aa-9bc6-b2b58a6a0ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:29.736124 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c0eb04e0-4b3e-48aa-9bc6-b2b58a6a0ac7 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:29.737770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c0eb04e0-4b3e-48aa-9bc6-b2b58a6a0ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:29.745084 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 993/2032] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:29.748377 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1040/2033] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:29.756655 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70241a13-d19f-4f1b-b2ea-9864de7093ec None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:29.759163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-70241a13-d19f-4f1b-b2ea-9864de7093ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:29.763538 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e6bad-6c75-4b91-8dc6-83cfbea6378a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:29.764046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e6bad-6c75-4b91-8dc6-83cfbea6378a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:29.764490 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e6bad-6c75-4b91-8dc6-83cfbea6378a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:29.803865 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-335e6bad-6c75-4b91-8dc6-83cfbea6378a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:29.805163 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-335e6bad-6c75-4b91-8dc6-83cfbea6378a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:29.825475 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1041/2034] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:29.829924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e6bad-6c75-4b91-8dc6-83cfbea6378a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:29.841359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e6bad-6c75-4b91-8dc6-83cfbea6378a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:29.857401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bbb29-d2f0-44e0-95b5-6dee437496a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:29.858117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bbb29-d2f0-44e0-95b5-6dee437496a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:29.858408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bbb29-d2f0-44e0-95b5-6dee437496a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:29.858690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bbb29-d2f0-44e0-95b5-6dee437496a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:29.858940 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70241a13-d19f-4f1b-b2ea-9864de7093ec None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:29.859481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790bbb29-d2f0-44e0-95b5-6dee437496a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:43:29.859789 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790bbb29-d2f0-44e0-95b5-6dee437496a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:29.861442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790bbb29-d2f0-44e0-95b5-6dee437496a9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:29.865333 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2dbbd4-94ce-4cfd-929e-e46180897857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:29.866629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2dbbd4-94ce-4cfd-929e-e46180897857 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:29.867178 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2dbbd4-94ce-4cfd-929e-e46180897857 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:29.867603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2dbbd4-94ce-4cfd-929e-e46180897857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:29.875430 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1042/2035] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:43:29.881513 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bbb29-d2f0-44e0-95b5-6dee437496a9 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:29.883544 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2dbbd4-94ce-4cfd-929e-e46180897857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:29.885174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2dbbd4-94ce-4cfd-929e-e46180897857 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:29.897643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 994/2036] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:29.898942 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bbb29-d2f0-44e0-95b5-6dee437496a9 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:29.906356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453b6cc-f570-40d3-b0b1-86b7c75f408f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:29.907868 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453b6cc-f570-40d3-b0b1-86b7c75f408f admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:29.908666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453b6cc-f570-40d3-b0b1-86b7c75f408f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:29.911156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453b6cc-f570-40d3-b0b1-86b7c75f408f admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:29.911944 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453b6cc-f570-40d3-b0b1-86b7c75f408f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:29.916100 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453b6cc-f570-40d3-b0b1-86b7c75f408f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:29.970673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1043/2037] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:29 2022] POST /identity/v3/projects => generated 383 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:43:29.975175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2dbbd4-94ce-4cfd-929e-e46180897857 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:29.992019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2dbbd4-94ce-4cfd-929e-e46180897857 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:30.116622 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ca82a-652e-4354-933e-e0162f61dc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:30.120034 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ca82a-652e-4354-933e-e0162f61dc2b admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:30.120955 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ca82a-652e-4354-933e-e0162f61dc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:30.122003 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ca82a-652e-4354-933e-e0162f61dc2b admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:30.123117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ca82a-652e-4354-933e-e0162f61dc2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:30.150368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ca82a-652e-4354-933e-e0162f61dc2b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:30.194033 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 995/2038] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:29 2022] POST /identity/v3/users => generated 292 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:30.198528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453b6cc-f570-40d3-b0b1-86b7c75f408f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:30.207617 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:30.212096 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453b6cc-f570-40d3-b0b1-86b7c75f408f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:30.221420 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a474a-0731-438a-b9dc-e2c49849035f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:30.221681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a474a-0731-438a-b9dc-e2c49849035f admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:30.222234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a474a-0731-438a-b9dc-e2c49849035f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:30.222601 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a474a-0731-438a-b9dc-e2c49849035f admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:30.223818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a474a-0731-438a-b9dc-e2c49849035f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:30.226619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a474a-0731-438a-b9dc-e2c49849035f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:30.241934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1044/2039] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:30.245872 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ca82a-652e-4354-933e-e0162f61dc2b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:30.264519 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ca82a-652e-4354-933e-e0162f61dc2b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:30.269786 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ceacb4-504c-4b52-859a-efdcd30bad83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:30.270210 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ceacb4-504c-4b52-859a-efdcd30bad83 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:30.270498 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ceacb4-504c-4b52-859a-efdcd30bad83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:30.270780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ceacb4-504c-4b52-859a-efdcd30bad83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:30.271524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ceacb4-504c-4b52-859a-efdcd30bad83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:30.273432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ceacb4-504c-4b52-859a-efdcd30bad83 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:30.286427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 996/2040] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:30.290041 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a474a-0731-438a-b9dc-e2c49849035f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:30.305292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a474a-0731-438a-b9dc-e2c49849035f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:30.312362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2e909-7292-437a-9b74-792f1d49db25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:30.313666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2e909-7292-437a-9b74-792f1d49db25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:30.314177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2e909-7292-437a-9b74-792f1d49db25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:30.314622 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2e909-7292-437a-9b74-792f1d49db25 admin admin] PATH_INFO: `/v3/projects/d42e7529420f4b69a691768ec0ca1b25/users/aba49623ff4749279e52d22821b07261/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:30.331969 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2e909-7292-437a-9b74-792f1d49db25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d42e7529420f4b69a691768ec0ca1b25, user_id=aba49623ff4749279e52d22821b07261, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:30.335071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2e909-7292-437a-9b74-792f1d49db25 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:30.353418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1045/2041] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:30 2022] PUT /identity/v3/projects/d42e7529420f4b69a691768ec0ca1b25/users/aba49623ff4749279e52d22821b07261/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:30.365536 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccd6a6-eb72-4bc2-a5d6-0cb509dbea83 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:30.366294 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccd6a6-eb72-4bc2-a5d6-0cb509dbea83 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:30.366977 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccd6a6-eb72-4bc2-a5d6-0cb509dbea83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:30.403339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-afccd6a6-eb72-4bc2-a5d6-0cb509dbea83 None None] MFA Rules not processed for user `aba49623ff4749279e52d22821b07261`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:30.405366 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-afccd6a6-eb72-4bc2-a5d6-0cb509dbea83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:30.414183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:30.542614 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 997/2042] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:30.544942 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2e909-7292-437a-9b74-792f1d49db25 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:30.587443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccd6a6-eb72-4bc2-a5d6-0cb509dbea83 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:30.588986 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-afccd6a6-eb72-4bc2-a5d6-0cb509dbea83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:30.653252 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2e909-7292-437a-9b74-792f1d49db25 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:30.659667 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd314b-9f33-4490-bc53-b9489cdd17a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:30.661026 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd314b-9f33-4490-bc53-b9489cdd17a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:30.661414 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd314b-9f33-4490-bc53-b9489cdd17a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:30.661828 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd314b-9f33-4490-bc53-b9489cdd17a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:30.662786 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd314b-9f33-4490-bc53-b9489cdd17a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:30.665794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd314b-9f33-4490-bc53-b9489cdd17a0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:30.706989 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccd6a6-eb72-4bc2-a5d6-0cb509dbea83 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:30.708212 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1046/2043] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:30 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:30.710069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e2f3f-c3d9-4bd3-a1e2-32b6b23d8a7f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:30.711548 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e2f3f-c3d9-4bd3-a1e2-32b6b23d8a7f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:30.711849 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e2f3f-c3d9-4bd3-a1e2-32b6b23d8a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:30.747462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-985e2f3f-c3d9-4bd3-a1e2-32b6b23d8a7f None None] MFA Rules not processed for user `ebc1b31030634b5eb17d3549959f5507`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:30.748753 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-985e2f3f-c3d9-4bd3-a1e2-32b6b23d8a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:30.795145 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:30.795714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:30.798569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:30.798912 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f admin admin] PATH_INFO: `/v3/projects/9270edf0542146ba81bbc81ca8a3df53` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:30.800406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9270edf0542146ba81bbc81ca8a3df53)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:30.802492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:30.878751 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1047/2044] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:30.879958 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e2f3f-c3d9-4bd3-a1e2-32b6b23d8a7f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:30.901140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9270edf0542146ba81bbc81ca8a3df53', 'request_id': 'req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:30.904377 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e2f3f-c3d9-4bd3-a1e2-32b6b23d8a7f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:30.909709 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:30.912504 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9270edf0542146ba81bbc81ca8a3df53', 'request_id': 'req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:30.928466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9270edf0542146ba81bbc81ca8a3df53', 'request_id': 'req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:30.960895 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f admin admin] The token cache is being invalidate because project 9270edf0542146ba81bbc81ca8a3df53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:30.961191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:30.963794 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:30.966076 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:30.969606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 998/2045] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:30 2022] DELETE /identity/v3/projects/9270edf0542146ba81bbc81ca8a3df53 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:43:30.971640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:30.980649 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:30.996142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d40cd4-eb31-4b46-9207-92fa9e890186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:30.997026 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d40cd4-eb31-4b46-9207-92fa9e890186 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:30.997469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d40cd4-eb31-4b46-9207-92fa9e890186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:30.997800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d40cd4-eb31-4b46-9207-92fa9e890186 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:30.998810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d40cd4-eb31-4b46-9207-92fa9e890186 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:31.002220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d40cd4-eb31-4b46-9207-92fa9e890186 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:31.042499 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1048/2046] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:30 2022] POST /identity/v3/users => generated 299 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:31.042499 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d40cd4-eb31-4b46-9207-92fa9e890186 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:31.043417 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-68d40cd4-eb31-4b46-9207-92fa9e890186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:31.074100 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d40cd4-eb31-4b46-9207-92fa9e890186 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:31.081011 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bea2e-00c6-46e7-96b3-d2a28b62eb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:31.081916 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bea2e-00c6-46e7-96b3-d2a28b62eb9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:31.082794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bea2e-00c6-46e7-96b3-d2a28b62eb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:31.083119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bea2e-00c6-46e7-96b3-d2a28b62eb9f admin admin] PATH_INFO: `/v3/users/097aef38d7a1446a80111e8d7b8e40a7` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:31.084994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399bea2e-00c6-46e7-96b3-d2a28b62eb9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=097aef38d7a1446a80111e8d7b8e40a7)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:31.087455 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399bea2e-00c6-46e7-96b3-d2a28b62eb9f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:31.146393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a1a9c-78d4-4be9-9296-5b494b817c8f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:31.148167 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-399bea2e-00c6-46e7-96b3-d2a28b62eb9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '097aef38d7a1446a80111e8d7b8e40a7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:31.149088 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-399bea2e-00c6-46e7-96b3-d2a28b62eb9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '097aef38d7a1446a80111e8d7b8e40a7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:31.149652 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-399bea2e-00c6-46e7-96b3-d2a28b62eb9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '097aef38d7a1446a80111e8d7b8e40a7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:31.152966 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212df26-6fe4-4339-8fb1-75b2e2f3e92f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:31.153576 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212df26-6fe4-4339-8fb1-75b2e2f3e92f service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:31.153864 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212df26-6fe4-4339-8fb1-75b2e2f3e92f service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:31.154176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212df26-6fe4-4339-8fb1-75b2e2f3e92f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:31.156134 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0212df26-6fe4-4339-8fb1-75b2e2f3e92f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:31.164980 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-399bea2e-00c6-46e7-96b3-d2a28b62eb9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '097aef38d7a1446a80111e8d7b8e40a7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:31.172469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-399bea2e-00c6-46e7-96b3-d2a28b62eb9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '097aef38d7a1446a80111e8d7b8e40a7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:31.193356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1049/2047] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:31 2022] DELETE /identity/v3/users/097aef38d7a1446a80111e8d7b8e40a7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:31.197116 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bea2e-00c6-46e7-96b3-d2a28b62eb9f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:31.201029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-399bea2e-00c6-46e7-96b3-d2a28b62eb9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:31.283533 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212df26-6fe4-4339-8fb1-75b2e2f3e92f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:31.285451 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212df26-6fe4-4339-8fb1-75b2e2f3e92f service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:31.298482 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 999/2048] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:31.299770 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7735b17-0afc-42ab-8c7a-5c84c09413a9 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:31.300062 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7735b17-0afc-42ab-8c7a-5c84c09413a9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:31.300414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7735b17-0afc-42ab-8c7a-5c84c09413a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:31.357029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bea2e-00c6-46e7-96b3-d2a28b62eb9f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:31.370109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f09c13-5f58-49d8-a45b-e5f2c3ae6878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:31.370936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f09c13-5f58-49d8-a45b-e5f2c3ae6878 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:31.371224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f09c13-5f58-49d8-a45b-e5f2c3ae6878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:31.371501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f09c13-5f58-49d8-a45b-e5f2c3ae6878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:31.372264 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f09c13-5f58-49d8-a45b-e5f2c3ae6878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:31.373881 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f09c13-5f58-49d8-a45b-e5f2c3ae6878 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:31.399703 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1050/2049] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:31.400724 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f09c13-5f58-49d8-a45b-e5f2c3ae6878 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:31.408993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-c7735b17-0afc-42ab-8c7a-5c84c09413a9 None None] MFA Rules not processed for user `287144c6346a43d09cdbae463ca9a961`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:31.412572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c7735b17-0afc-42ab-8c7a-5c84c09413a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:31.423487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f09c13-5f58-49d8-a45b-e5f2c3ae6878 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:31.440805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cabc67-b589-46f2-90c6-2ebf87d80753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:31.442786 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cabc67-b589-46f2-90c6-2ebf87d80753 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:31.443087 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cabc67-b589-46f2-90c6-2ebf87d80753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:31.443371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cabc67-b589-46f2-90c6-2ebf87d80753 admin admin] PATH_INFO: `/v3/projects/21ca3b9964584e7a8d88df373d824038/users/13b82548e9314119af0eab8733d238f7/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:31.470849 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cabc67-b589-46f2-90c6-2ebf87d80753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ca3b9964584e7a8d88df373d824038, user_id=13b82548e9314119af0eab8733d238f7, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:31.474260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cabc67-b589-46f2-90c6-2ebf87d80753 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:31.495564 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1051/2050] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:31 2022] PUT /identity/v3/projects/21ca3b9964584e7a8d88df373d824038/users/13b82548e9314119af0eab8733d238f7/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:31.499935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cabc67-b589-46f2-90c6-2ebf87d80753 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:31.526698 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:31.586073 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1000/2051] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:31.594196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7735b17-0afc-42ab-8c7a-5c84c09413a9 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:31.595984 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c7735b17-0afc-42ab-8c7a-5c84c09413a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:31.624002 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:31.630718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cabc67-b589-46f2-90c6-2ebf87d80753 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:31.640406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4a7bd-bf51-409d-99f4-428e244367f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:31.641815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4a7bd-bf51-409d-99f4-428e244367f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:31.642377 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4a7bd-bf51-409d-99f4-428e244367f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:31.643413 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4a7bd-bf51-409d-99f4-428e244367f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:31.644268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4a7bd-bf51-409d-99f4-428e244367f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:31.648411 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4a7bd-bf51-409d-99f4-428e244367f4 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:31.660665 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1052/2052] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:31.666015 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4a7bd-bf51-409d-99f4-428e244367f4 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:31.692252 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4a7bd-bf51-409d-99f4-428e244367f4 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:31.704429 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95120f4-2722-466c-9cd4-5dbce4974472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:31.705608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95120f4-2722-466c-9cd4-5dbce4974472 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:31.706150 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95120f4-2722-466c-9cd4-5dbce4974472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:31.708449 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95120f4-2722-466c-9cd4-5dbce4974472 admin admin] PATH_INFO: `/v3/projects/21ca3b9964584e7a8d88df373d824038/users/13b82548e9314119af0eab8733d238f7/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:31.712077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95120f4-2722-466c-9cd4-5dbce4974472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ca3b9964584e7a8d88df373d824038, user_id=13b82548e9314119af0eab8733d238f7, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:31.716871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95120f4-2722-466c-9cd4-5dbce4974472 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:31.721152 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7735b17-0afc-42ab-8c7a-5c84c09413a9 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:31.746258 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1053/2053] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:31 2022] PUT /identity/v3/projects/21ca3b9964584e7a8d88df373d824038/users/13b82548e9314119af0eab8733d238f7/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:31.747570 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95120f4-2722-466c-9cd4-5dbce4974472 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:31.749732 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a95120f4-2722-466c-9cd4-5dbce4974472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:31.825159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efaa782-8387-4dd0-9e7a-7fb621dae53d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:31.826101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efaa782-8387-4dd0-9e7a-7fb621dae53d service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:31.826425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efaa782-8387-4dd0-9e7a-7fb621dae53d service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:31.827234 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efaa782-8387-4dd0-9e7a-7fb621dae53d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:31.871208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95120f4-2722-466c-9cd4-5dbce4974472 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:31.881874 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a1777-9031-45c8-ae0c-907fdf0dff17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:31.883169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a1777-9031-45c8-ae0c-907fdf0dff17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:31.883169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a1777-9031-45c8-ae0c-907fdf0dff17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:31.883389 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a1777-9031-45c8-ae0c-907fdf0dff17 admin admin] PATH_INFO: `/v3/projects/22a2037e8e3640eb8ed09e352d112796` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:31.886270 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a1777-9031-45c8-ae0c-907fdf0dff17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22a2037e8e3640eb8ed09e352d112796)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:31.891274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a1777-9031-45c8-ae0c-907fdf0dff17 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:31.909734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efaa782-8387-4dd0-9e7a-7fb621dae53d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:31.912529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efaa782-8387-4dd0-9e7a-7fb621dae53d service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:31.925259 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1001/2054] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:31.927193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264eae4-a80a-4993-843d-5fd45f6613e6 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:31.927616 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264eae4-a80a-4993-843d-5fd45f6613e6 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:31.927900 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264eae4-a80a-4993-843d-5fd45f6613e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:31.958369 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a72a1777-9031-45c8-ae0c-907fdf0dff17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22a2037e8e3640eb8ed09e352d112796', 'request_id': 'req-a72a1777-9031-45c8-ae0c-907fdf0dff17'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:31.966119 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-6264eae4-a80a-4993-843d-5fd45f6613e6 None None] MFA Rules not processed for user `13b82548e9314119af0eab8733d238f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:31.967574 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6264eae4-a80a-4993-843d-5fd45f6613e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:31.983659 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a72a1777-9031-45c8-ae0c-907fdf0dff17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22a2037e8e3640eb8ed09e352d112796', 'request_id': 'req-a72a1777-9031-45c8-ae0c-907fdf0dff17'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:32.026345 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a72a1777-9031-45c8-ae0c-907fdf0dff17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22a2037e8e3640eb8ed09e352d112796', 'request_id': 'req-a72a1777-9031-45c8-ae0c-907fdf0dff17'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:32.061325 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a72a1777-9031-45c8-ae0c-907fdf0dff17 admin admin] The token cache is being invalidate because project 22a2037e8e3640eb8ed09e352d112796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:32.061934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a72a1777-9031-45c8-ae0c-907fdf0dff17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:32.063613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a72a1777-9031-45c8-ae0c-907fdf0dff17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:32.070012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1054/2055] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:31 2022] DELETE /identity/v3/projects/22a2037e8e3640eb8ed09e352d112796 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:32.076213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51400b90-ab8a-47f9-9511-5a886c200677 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:32.077360 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51400b90-ab8a-47f9-9511-5a886c200677 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:32.077851 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51400b90-ab8a-47f9-9511-5a886c200677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:32.119949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-51400b90-ab8a-47f9-9511-5a886c200677 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:32.120980 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1002/2056] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:32.122461 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-51400b90-ab8a-47f9-9511-5a886c200677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:32.147524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264eae4-a80a-4993-843d-5fd45f6613e6 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:32.149182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6264eae4-a80a-4993-843d-5fd45f6613e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:32.167322 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:32.200077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:32.296464 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264eae4-a80a-4993-843d-5fd45f6613e6 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:32.302113 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151446eb-f29c-4fb0-9f30-287195d91b0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:32.302933 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151446eb-f29c-4fb0-9f30-287195d91b0c service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:32.303223 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151446eb-f29c-4fb0-9f30-287195d91b0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:32.303804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151446eb-f29c-4fb0-9f30-287195d91b0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:32.306720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-151446eb-f29c-4fb0-9f30-287195d91b0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:32.316047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1055/2057] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:43:32.324824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259eaafe-add7-42bf-a43e-c3855d265b65 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:32.325097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259eaafe-add7-42bf-a43e-c3855d265b65 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:32.325470 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259eaafe-add7-42bf-a43e-c3855d265b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:32.356246 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-259eaafe-add7-42bf-a43e-c3855d265b65 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:32.357714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-259eaafe-add7-42bf-a43e-c3855d265b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:32.385665 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1056/2058] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:32.386628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259eaafe-add7-42bf-a43e-c3855d265b65 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:32.402928 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259eaafe-add7-42bf-a43e-c3855d265b65 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:32.408174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e80a4-e3f2-4e4d-bffe-fae646007a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:32.408859 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e80a4-e3f2-4e4d-bffe-fae646007a88 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:32.409141 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e80a4-e3f2-4e4d-bffe-fae646007a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:32.409456 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e80a4-e3f2-4e4d-bffe-fae646007a88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:32.410306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e80a4-e3f2-4e4d-bffe-fae646007a88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:43:32.411111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e80a4-e3f2-4e4d-bffe-fae646007a88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:32.412745 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e80a4-e3f2-4e4d-bffe-fae646007a88 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:32.419695 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151446eb-f29c-4fb0-9f30-287195d91b0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:32.421371 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151446eb-f29c-4fb0-9f30-287195d91b0c service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:32.431702 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1057/2059] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:43:32.436642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e80a4-e3f2-4e4d-bffe-fae646007a88 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:32.450413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1003/2060] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:43:32.451009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e80a4-e3f2-4e4d-bffe-fae646007a88 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:32.458194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac1e46-3b89-4511-8b61-d4a68fc52976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:32.459259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac1e46-3b89-4511-8b61-d4a68fc52976 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:32.459546 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac1e46-3b89-4511-8b61-d4a68fc52976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:32.459821 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac1e46-3b89-4511-8b61-d4a68fc52976 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:32.460652 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac1e46-3b89-4511-8b61-d4a68fc52976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:32.463324 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac1e46-3b89-4511-8b61-d4a68fc52976 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:32.538964 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1058/2061] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:32 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:32.540640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151446eb-f29c-4fb0-9f30-287195d91b0c service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:32.556039 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151446eb-f29c-4fb0-9f30-287195d91b0c service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:32.627088 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e5531-e77c-4d4c-bd49-0774cf7f7e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:32.628282 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e5531-e77c-4d4c-bd49-0774cf7f7e61 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:32.628740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e5531-e77c-4d4c-bd49-0774cf7f7e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:32.629589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e5531-e77c-4d4c-bd49-0774cf7f7e61 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:32.630346 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e5531-e77c-4d4c-bd49-0774cf7f7e61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:32.632552 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e5531-e77c-4d4c-bd49-0774cf7f7e61 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:32.665347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1004/2062] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:32 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:43:32.669571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac1e46-3b89-4511-8b61-d4a68fc52976 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:32.681548 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac1e46-3b89-4511-8b61-d4a68fc52976 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:32.686292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edab6fa-ab52-4980-8a4b-d9ffa90fac44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:32.686960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edab6fa-ab52-4980-8a4b-d9ffa90fac44 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:32.687248 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edab6fa-ab52-4980-8a4b-d9ffa90fac44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:32.687786 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edab6fa-ab52-4980-8a4b-d9ffa90fac44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:32.688243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edab6fa-ab52-4980-8a4b-d9ffa90fac44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:32.689914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edab6fa-ab52-4980-8a4b-d9ffa90fac44 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:32.703318 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1059/2063] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:32.707983 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e5531-e77c-4d4c-bd49-0774cf7f7e61 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:32.720010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e5531-e77c-4d4c-bd49-0774cf7f7e61 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:32.725388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9814ec1-9b8b-4d52-ae6f-4191d75e56ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:32.726122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9814ec1-9b8b-4d52-ae6f-4191d75e56ea admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:32.726390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9814ec1-9b8b-4d52-ae6f-4191d75e56ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:32.726660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9814ec1-9b8b-4d52-ae6f-4191d75e56ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:32.727404 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9814ec1-9b8b-4d52-ae6f-4191d75e56ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:32.729241 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9814ec1-9b8b-4d52-ae6f-4191d75e56ea admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:32.740110 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1005/2064] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:43:32.748083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edab6fa-ab52-4980-8a4b-d9ffa90fac44 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:32.767718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edab6fa-ab52-4980-8a4b-d9ffa90fac44 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:32.774674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8940e4-3317-4af9-98d5-e631fb9b81f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:32.775728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8940e4-3317-4af9-98d5-e631fb9b81f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:32.776046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8940e4-3317-4af9-98d5-e631fb9b81f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:32.776364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8940e4-3317-4af9-98d5-e631fb9b81f4 admin admin] PATH_INFO: `/v3/projects/fe86eb3cbcdd4c5b9154b9275a10e4ac/users/4f9754cf9b28467d9243b75bbea65d64/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:32.794111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8940e4-3317-4af9-98d5-e631fb9b81f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe86eb3cbcdd4c5b9154b9275a10e4ac, user_id=4f9754cf9b28467d9243b75bbea65d64, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:32.797369 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8940e4-3317-4af9-98d5-e631fb9b81f4 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:32.814226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1060/2065] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:32 2022] PUT /identity/v3/projects/fe86eb3cbcdd4c5b9154b9275a10e4ac/users/4f9754cf9b28467d9243b75bbea65d64/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:32.825868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b2b8c-580a-4cc2-9a82-ac89f56ffa38 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:32.826346 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b2b8c-580a-4cc2-9a82-ac89f56ffa38 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:32.826651 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b2b8c-580a-4cc2-9a82-ac89f56ffa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:32.882007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-c94b2b8c-580a-4cc2-9a82-ac89f56ffa38 None None] MFA Rules not processed for user `4f9754cf9b28467d9243b75bbea65d64`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:32.883488 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c94b2b8c-580a-4cc2-9a82-ac89f56ffa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:32.933573 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:32.985549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1006/2066] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:32.990620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8940e4-3317-4af9-98d5-e631fb9b81f4 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:33.063681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8940e4-3317-4af9-98d5-e631fb9b81f4 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:33.069015 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5604acb-e044-4cbe-9e76-18123efeca52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:33.069673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5604acb-e044-4cbe-9e76-18123efeca52 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:33.069996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5604acb-e044-4cbe-9e76-18123efeca52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:33.070274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5604acb-e044-4cbe-9e76-18123efeca52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:33.070975 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5604acb-e044-4cbe-9e76-18123efeca52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:33.072895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5604acb-e044-4cbe-9e76-18123efeca52 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:33.104924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1061/2067] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:32 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:33.105739 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b2b8c-580a-4cc2-9a82-ac89f56ffa38 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:33.117104 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b2b8c-580a-4cc2-9a82-ac89f56ffa38 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:33.179227 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2544e24-29eb-444f-917b-54c77ef1882d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:33.179908 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2544e24-29eb-444f-917b-54c77ef1882d admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:33.180193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2544e24-29eb-444f-917b-54c77ef1882d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:33.180501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2544e24-29eb-444f-917b-54c77ef1882d admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:33.181190 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2544e24-29eb-444f-917b-54c77ef1882d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:33.183343 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2544e24-29eb-444f-917b-54c77ef1882d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:33.205604 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1007/2068] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:33 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:43:33.209794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5604acb-e044-4cbe-9e76-18123efeca52 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:33.220935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5604acb-e044-4cbe-9e76-18123efeca52 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:33.226825 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d97a8-4cae-4ec4-9ebc-ee72ab9e8092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:33.227581 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d97a8-4cae-4ec4-9ebc-ee72ab9e8092 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:33.227865 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d97a8-4cae-4ec4-9ebc-ee72ab9e8092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:33.228143 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d97a8-4cae-4ec4-9ebc-ee72ab9e8092 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:33.229099 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d97a8-4cae-4ec4-9ebc-ee72ab9e8092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:33.231387 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d97a8-4cae-4ec4-9ebc-ee72ab9e8092 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:33.243697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1062/2069] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:33.248774 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2544e24-29eb-444f-917b-54c77ef1882d admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:33.266660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2544e24-29eb-444f-917b-54c77ef1882d admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:33.271610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308487b3-ef5e-4fc3-902e-4ea0244d7afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:33.272492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308487b3-ef5e-4fc3-902e-4ea0244d7afb admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:33.272780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308487b3-ef5e-4fc3-902e-4ea0244d7afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:33.273057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308487b3-ef5e-4fc3-902e-4ea0244d7afb admin admin] PATH_INFO: `/v3/projects/52402865a38f4b24946f9b078b70c4a3/users/76374aa5e0164284a887774e7be4223a/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:33.289040 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308487b3-ef5e-4fc3-902e-4ea0244d7afb admin admin] RBAC: Authorizing `identity:create_grant(project_id=52402865a38f4b24946f9b078b70c4a3, user_id=76374aa5e0164284a887774e7be4223a, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:33.291808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308487b3-ef5e-4fc3-902e-4ea0244d7afb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:33.308617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1008/2070] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:33 2022] PUT /identity/v3/projects/52402865a38f4b24946f9b078b70c4a3/users/76374aa5e0164284a887774e7be4223a/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:33.312342 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d97a8-4cae-4ec4-9ebc-ee72ab9e8092 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:33.323310 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d97a8-4cae-4ec4-9ebc-ee72ab9e8092 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:33.349242 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:33.416696 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b005585-46a5-4b37-88b6-178ca5734fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:33.417800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b005585-46a5-4b37-88b6-178ca5734fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:33.418365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b005585-46a5-4b37-88b6-178ca5734fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:33.418867 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b005585-46a5-4b37-88b6-178ca5734fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:33.419859 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b005585-46a5-4b37-88b6-178ca5734fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:33.421881 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b005585-46a5-4b37-88b6-178ca5734fa6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:33.437056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1063/2071] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:43:33.442438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308487b3-ef5e-4fc3-902e-4ea0244d7afb admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:33.454686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308487b3-ef5e-4fc3-902e-4ea0244d7afb admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:33.462874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5cd7b-d234-46c3-87d9-0fc621887d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:33.463794 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5cd7b-d234-46c3-87d9-0fc621887d8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:33.464110 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5cd7b-d234-46c3-87d9-0fc621887d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:33.464385 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5cd7b-d234-46c3-87d9-0fc621887d8f admin admin] PATH_INFO: `/v3/projects/52402865a38f4b24946f9b078b70c4a3/users/76374aa5e0164284a887774e7be4223a/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:33.468542 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5cd7b-d234-46c3-87d9-0fc621887d8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=52402865a38f4b24946f9b078b70c4a3, user_id=76374aa5e0164284a887774e7be4223a, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:33.471265 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5cd7b-d234-46c3-87d9-0fc621887d8f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:33.484460 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1009/2072] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:33 2022] PUT /identity/v3/projects/52402865a38f4b24946f9b078b70c4a3/users/76374aa5e0164284a887774e7be4223a/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:33.497463 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cbda1-1873-4b8e-bc32-60703a6a4ad5 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:33.497721 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cbda1-1873-4b8e-bc32-60703a6a4ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:33.498556 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cbda1-1873-4b8e-bc32-60703a6a4ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:33.528764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-772cbda1-1873-4b8e-bc32-60703a6a4ad5 None None] MFA Rules not processed for user `76374aa5e0164284a887774e7be4223a`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:33.530297 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-772cbda1-1873-4b8e-bc32-60703a6a4ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:33.631707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bed6db-e2c1-4f8f-b94a-6b228dfb96c8 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:33.632294 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bed6db-e2c1-4f8f-b94a-6b228dfb96c8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:33.632809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bed6db-e2c1-4f8f-b94a-6b228dfb96c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:33.660721 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1064/2073] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:33.664426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e5bed6db-e2c1-4f8f-b94a-6b228dfb96c8 None None] MFA Rules not processed for user `287144c6346a43d09cdbae463ca9a961`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:33.666114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e5bed6db-e2c1-4f8f-b94a-6b228dfb96c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:33.673850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cbda1-1873-4b8e-bc32-60703a6a4ad5 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:33.749480 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cbda1-1873-4b8e-bc32-60703a6a4ad5 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:33.762811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c42ea7-be6e-4e1e-97b1-6702eab51925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:33.763474 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c42ea7-be6e-4e1e-97b1-6702eab51925 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:33.763757 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c42ea7-be6e-4e1e-97b1-6702eab51925 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:33.764030 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c42ea7-be6e-4e1e-97b1-6702eab51925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:33.767075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1010/2074] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:33.785976 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c42ea7-be6e-4e1e-97b1-6702eab51925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:33.787457 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c42ea7-be6e-4e1e-97b1-6702eab51925 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:33.814280 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1065/2075] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:43:33.902800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bed6db-e2c1-4f8f-b94a-6b228dfb96c8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:33.934102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:33.985993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bed6db-e2c1-4f8f-b94a-6b228dfb96c8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:33.991970 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e90b8-5ae9-4f38-b583-8e44a3788c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:33.993126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e90b8-5ae9-4f38-b583-8e44a3788c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:33.993419 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e90b8-5ae9-4f38-b583-8e44a3788c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:33.994585 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e90b8-5ae9-4f38-b583-8e44a3788c8b admin admin] PATH_INFO: `/v3/users/4f9754cf9b28467d9243b75bbea65d64` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:33.996580 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e90b8-5ae9-4f38-b583-8e44a3788c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f9754cf9b28467d9243b75bbea65d64)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:33.999084 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e90b8-5ae9-4f38-b583-8e44a3788c8b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:34.088028 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-645e90b8-5ae9-4f38-b583-8e44a3788c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f9754cf9b28467d9243b75bbea65d64'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:34.103213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-645e90b8-5ae9-4f38-b583-8e44a3788c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f9754cf9b28467d9243b75bbea65d64'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:34.116304 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-645e90b8-5ae9-4f38-b583-8e44a3788c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f9754cf9b28467d9243b75bbea65d64'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:34.117074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-645e90b8-5ae9-4f38-b583-8e44a3788c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f9754cf9b28467d9243b75bbea65d64'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:34.138659 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-645e90b8-5ae9-4f38-b583-8e44a3788c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f9754cf9b28467d9243b75bbea65d64'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:34.154248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1011/2076] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:33 2022] DELETE /identity/v3/users/4f9754cf9b28467d9243b75bbea65d64 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:34.158955 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f3f6e-c92a-4154-9f06-f6842d8d77e7 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:34.159251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f3f6e-c92a-4154-9f06-f6842d8d77e7 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:34.159533 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f3f6e-c92a-4154-9f06-f6842d8d77e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:34.179297 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ec2f3f6e-c92a-4154-9f06-f6842d8d77e7 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:34.180900 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ec2f3f6e-c92a-4154-9f06-f6842d8d77e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:34.241456 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:34.261718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1066/2077] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:34.274271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e90b8-5ae9-4f38-b583-8e44a3788c8b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:34.275759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-645e90b8-5ae9-4f38-b583-8e44a3788c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:34.361433 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e90b8-5ae9-4f38-b583-8e44a3788c8b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:34.366048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55397115-f28a-4481-ad00-5eb44cfff49e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:34.367788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55397115-f28a-4481-ad00-5eb44cfff49e service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:34.368374 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55397115-f28a-4481-ad00-5eb44cfff49e service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:34.368835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55397115-f28a-4481-ad00-5eb44cfff49e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:34.382733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55397115-f28a-4481-ad00-5eb44cfff49e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:34.384897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55397115-f28a-4481-ad00-5eb44cfff49e service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:34.405806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1012/2078] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:34.796855 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f3f6e-c92a-4154-9f06-f6842d8d77e7 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:34.798782 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ec2f3f6e-c92a-4154-9f06-f6842d8d77e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:34.818729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f3f6e-c92a-4154-9f06-f6842d8d77e7 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:34.825406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96933c7-d65e-4699-ae20-f14290763067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:34.826209 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96933c7-d65e-4699-ae20-f14290763067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:34.826523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96933c7-d65e-4699-ae20-f14290763067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:34.826730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96933c7-d65e-4699-ae20-f14290763067 admin admin] PATH_INFO: `/v3/projects/fe86eb3cbcdd4c5b9154b9275a10e4ac` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:34.828132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96933c7-d65e-4699-ae20-f14290763067 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe86eb3cbcdd4c5b9154b9275a10e4ac)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:34.830582 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96933c7-d65e-4699-ae20-f14290763067 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:34.907050 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a96933c7-d65e-4699-ae20-f14290763067 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe86eb3cbcdd4c5b9154b9275a10e4ac', 'request_id': 'req-a96933c7-d65e-4699-ae20-f14290763067'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:34.917042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a96933c7-d65e-4699-ae20-f14290763067 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe86eb3cbcdd4c5b9154b9275a10e4ac', 'request_id': 'req-a96933c7-d65e-4699-ae20-f14290763067'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:34.924921 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a96933c7-d65e-4699-ae20-f14290763067 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe86eb3cbcdd4c5b9154b9275a10e4ac', 'request_id': 'req-a96933c7-d65e-4699-ae20-f14290763067'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:34.940382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a96933c7-d65e-4699-ae20-f14290763067 admin admin] The token cache is being invalidate because project fe86eb3cbcdd4c5b9154b9275a10e4ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:34.940679 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a96933c7-d65e-4699-ae20-f14290763067 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:34.941380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a96933c7-d65e-4699-ae20-f14290763067 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:34.943042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1067/2079] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:34 2022] DELETE /identity/v3/projects/fe86eb3cbcdd4c5b9154b9275a10e4ac => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:34.947232 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55397115-f28a-4481-ad00-5eb44cfff49e service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:34.957839 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55397115-f28a-4481-ad00-5eb44cfff49e service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:35.074516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:35.090919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5ffe3-d85e-4ee9-b031-82eb62207689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:35.092693 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5ffe3-d85e-4ee9-b031-82eb62207689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:35.093191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5ffe3-d85e-4ee9-b031-82eb62207689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:35.093644 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5ffe3-d85e-4ee9-b031-82eb62207689 admin admin] PATH_INFO: `/v3/users/76374aa5e0164284a887774e7be4223a` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:35.095440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5ffe3-d85e-4ee9-b031-82eb62207689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76374aa5e0164284a887774e7be4223a)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:35.098337 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5ffe3-d85e-4ee9-b031-82eb62207689 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:35.183089 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0de5ffe3-d85e-4ee9-b031-82eb62207689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76374aa5e0164284a887774e7be4223a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:35.198604 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0de5ffe3-d85e-4ee9-b031-82eb62207689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76374aa5e0164284a887774e7be4223a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:35.206404 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0de5ffe3-d85e-4ee9-b031-82eb62207689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76374aa5e0164284a887774e7be4223a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:35.208132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0de5ffe3-d85e-4ee9-b031-82eb62207689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76374aa5e0164284a887774e7be4223a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:35.216434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0de5ffe3-d85e-4ee9-b031-82eb62207689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76374aa5e0164284a887774e7be4223a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:35.223208 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1013/2080] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:34 2022] DELETE /identity/v3/users/76374aa5e0164284a887774e7be4223a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:35.607921 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96933c7-d65e-4699-ae20-f14290763067 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:35.609261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a96933c7-d65e-4699-ae20-f14290763067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:35.635114 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96933c7-d65e-4699-ae20-f14290763067 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:35.718572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792625f-e6cc-427a-865d-47acc68310ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:35.719468 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792625f-e6cc-427a-865d-47acc68310ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:35.719868 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792625f-e6cc-427a-865d-47acc68310ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:35.720155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792625f-e6cc-427a-865d-47acc68310ce admin admin] PATH_INFO: `/v3/projects/52402865a38f4b24946f9b078b70c4a3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:35.721470 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792625f-e6cc-427a-865d-47acc68310ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=52402865a38f4b24946f9b078b70c4a3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:35.724071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792625f-e6cc-427a-865d-47acc68310ce admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:35.796731 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7792625f-e6cc-427a-865d-47acc68310ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52402865a38f4b24946f9b078b70c4a3', 'request_id': 'req-7792625f-e6cc-427a-865d-47acc68310ce'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:35.808845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7792625f-e6cc-427a-865d-47acc68310ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52402865a38f4b24946f9b078b70c4a3', 'request_id': 'req-7792625f-e6cc-427a-865d-47acc68310ce'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:35.816918 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7792625f-e6cc-427a-865d-47acc68310ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52402865a38f4b24946f9b078b70c4a3', 'request_id': 'req-7792625f-e6cc-427a-865d-47acc68310ce'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:35.828940 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7792625f-e6cc-427a-865d-47acc68310ce admin admin] The token cache is being invalidate because project 52402865a38f4b24946f9b078b70c4a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:35.829227 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7792625f-e6cc-427a-865d-47acc68310ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:35.830694 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7792625f-e6cc-427a-865d-47acc68310ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:35.832570 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1068/2081] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:35 2022] DELETE /identity/v3/projects/52402865a38f4b24946f9b078b70c4a3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:35.837513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67c745-3541-4989-a48d-07ede519183c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:35.838829 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67c745-3541-4989-a48d-07ede519183c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:35.839179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67c745-3541-4989-a48d-07ede519183c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:35.871524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1b67c745-3541-4989-a48d-07ede519183c None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:35.873015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1b67c745-3541-4989-a48d-07ede519183c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:36.004179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1014/2082] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:36.013104 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cd969-03c9-42bc-ad5d-152fe3af1f08 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:36.013635 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cd969-03c9-42bc-ad5d-152fe3af1f08 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:36.014396 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cd969-03c9-42bc-ad5d-152fe3af1f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:36.024968 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:36.043681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9a3cd969-03c9-42bc-ad5d-152fe3af1f08 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:36.045567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9a3cd969-03c9-42bc-ad5d-152fe3af1f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:36.056988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1069/2083] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:36.061095 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67c745-3541-4989-a48d-07ede519183c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:36.071824 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67c745-3541-4989-a48d-07ede519183c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:36.077015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8a46a-30d3-41f4-b327-f604c5822e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:36.077899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8a46a-30d3-41f4-b327-f604c5822e1e admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:36.078361 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8a46a-30d3-41f4-b327-f604c5822e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:36.078636 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8a46a-30d3-41f4-b327-f604c5822e1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:36.079580 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8a46a-30d3-41f4-b327-f604c5822e1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:43:36.079891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8a46a-30d3-41f4-b327-f604c5822e1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:36.081511 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8a46a-30d3-41f4-b327-f604c5822e1e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:36.099575 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1015/2084] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:43:36.100987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cd969-03c9-42bc-ad5d-152fe3af1f08 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:36.122618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cd969-03c9-42bc-ad5d-152fe3af1f08 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:36.136922 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824c445-6e5e-40de-980f-16c5781a6c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:36.137605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824c445-6e5e-40de-980f-16c5781a6c60 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:36.137913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824c445-6e5e-40de-980f-16c5781a6c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:36.138240 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824c445-6e5e-40de-980f-16c5781a6c60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:36.138969 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f824c445-6e5e-40de-980f-16c5781a6c60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:36.140900 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f824c445-6e5e-40de-980f-16c5781a6c60 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:36.188323 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1070/2085] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:36 2022] POST /identity/v3/projects => generated 401 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:36.192705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8a46a-30d3-41f4-b327-f604c5822e1e admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:36.204170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8a46a-30d3-41f4-b327-f604c5822e1e admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:36.295422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:36.311920 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ad1fa-5f72-4582-bd0a-868c2d2e05f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:36.312625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ad1fa-5f72-4582-bd0a-868c2d2e05f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:36.312916 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ad1fa-5f72-4582-bd0a-868c2d2e05f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:36.313192 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ad1fa-5f72-4582-bd0a-868c2d2e05f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:36.314095 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ad1fa-5f72-4582-bd0a-868c2d2e05f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:36.316075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ad1fa-5f72-4582-bd0a-868c2d2e05f7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:36.358698 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1016/2086] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:36 2022] POST /identity/v3/users => generated 301 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:36.369262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824c445-6e5e-40de-980f-16c5781a6c60 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:36.383480 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824c445-6e5e-40de-980f-16c5781a6c60 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:36.399884 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cce7b-59c4-4a7a-801f-c5f8632997d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:36.401437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cce7b-59c4-4a7a-801f-c5f8632997d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:36.401992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cce7b-59c4-4a7a-801f-c5f8632997d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:36.402480 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cce7b-59c4-4a7a-801f-c5f8632997d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:36.404633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cce7b-59c4-4a7a-801f-c5f8632997d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:36.415103 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cce7b-59c4-4a7a-801f-c5f8632997d5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:36.450474 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1071/2087] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:36.452010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ad1fa-5f72-4582-bd0a-868c2d2e05f7 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:36.474441 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ad1fa-5f72-4582-bd0a-868c2d2e05f7 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:36.482396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ce360-2ee0-4883-baca-98709cc59244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:36.486714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ce360-2ee0-4883-baca-98709cc59244 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:36.487314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ce360-2ee0-4883-baca-98709cc59244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:36.487886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ce360-2ee0-4883-baca-98709cc59244 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:36.489222 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ce360-2ee0-4883-baca-98709cc59244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:36.492067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ce360-2ee0-4883-baca-98709cc59244 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:36.506978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1017/2088] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:36.511144 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cce7b-59c4-4a7a-801f-c5f8632997d5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:36.522934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cce7b-59c4-4a7a-801f-c5f8632997d5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:36.534716 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be32e7-6f18-44c0-b832-562a71a3587d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:36.535736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be32e7-6f18-44c0-b832-562a71a3587d admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:36.536010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be32e7-6f18-44c0-b832-562a71a3587d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:36.536263 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be32e7-6f18-44c0-b832-562a71a3587d admin admin] PATH_INFO: `/v3/projects/f8f8b0241d1f48b6b929be821b75765f/users/49188820bbeb44509a642ca4ba81fc2d/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:36.568874 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be32e7-6f18-44c0-b832-562a71a3587d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8f8b0241d1f48b6b929be821b75765f, user_id=49188820bbeb44509a642ca4ba81fc2d, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:36.578566 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be32e7-6f18-44c0-b832-562a71a3587d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:36.612334 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1072/2089] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:36 2022] PUT /identity/v3/projects/f8f8b0241d1f48b6b929be821b75765f/users/49188820bbeb44509a642ca4ba81fc2d/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:36.622776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb40ca-4fd5-43ba-bc5a-69a5c73fc5bf None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:36.623740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb40ca-4fd5-43ba-bc5a-69a5c73fc5bf None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:36.623740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb40ca-4fd5-43ba-bc5a-69a5c73fc5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:36.655399 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-fbbb40ca-4fd5-43ba-bc5a-69a5c73fc5bf None None] MFA Rules not processed for user `49188820bbeb44509a642ca4ba81fc2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:36.656639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fbbb40ca-4fd5-43ba-bc5a-69a5c73fc5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:36.756776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1018/2090] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:43:36.763272 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be32e7-6f18-44c0-b832-562a71a3587d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:36.888973 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be32e7-6f18-44c0-b832-562a71a3587d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:36.902741 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6a749-9a9e-4d7e-834b-7aae70791423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:36.903427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6a749-9a9e-4d7e-834b-7aae70791423 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:36.903714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6a749-9a9e-4d7e-834b-7aae70791423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:36.903990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6a749-9a9e-4d7e-834b-7aae70791423 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:36.904714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6a749-9a9e-4d7e-834b-7aae70791423 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:36.906688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6a749-9a9e-4d7e-834b-7aae70791423 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:36.936036 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1073/2091] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:36 2022] POST /identity/v3/projects => generated 401 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:36.940209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb40ca-4fd5-43ba-bc5a-69a5c73fc5bf None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:36.951821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb40ca-4fd5-43ba-bc5a-69a5c73fc5bf None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:37.017288 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a0959-d075-41f0-9b6f-0ebb9ca3dd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:37.018034 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a0959-d075-41f0-9b6f-0ebb9ca3dd3c admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:37.018323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a0959-d075-41f0-9b6f-0ebb9ca3dd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:37.018599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a0959-d075-41f0-9b6f-0ebb9ca3dd3c admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:37.019300 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a0959-d075-41f0-9b6f-0ebb9ca3dd3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:37.022260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a0959-d075-41f0-9b6f-0ebb9ca3dd3c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:37.055313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1019/2092] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:36 2022] POST /identity/v3/users => generated 307 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:37.059077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6a749-9a9e-4d7e-834b-7aae70791423 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:37.070220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6a749-9a9e-4d7e-834b-7aae70791423 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:37.075552 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94695ee-3945-486d-a0c3-c153d0260afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:37.076230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94695ee-3945-486d-a0c3-c153d0260afb admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:37.076514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94695ee-3945-486d-a0c3-c153d0260afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:37.076789 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94695ee-3945-486d-a0c3-c153d0260afb admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:37.077531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94695ee-3945-486d-a0c3-c153d0260afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:37.079198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94695ee-3945-486d-a0c3-c153d0260afb admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:37.080874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a0959-d075-41f0-9b6f-0ebb9ca3dd3c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:37.083291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9d7a0959-d075-41f0-9b6f-0ebb9ca3dd3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:37.094055 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1074/2093] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:37.098715 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94695ee-3945-486d-a0c3-c153d0260afb admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:37.109765 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94695ee-3945-486d-a0c3-c153d0260afb admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:37.115175 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1066a-4e23-4513-bd15-fdc6b866d151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:37.116076 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1066a-4e23-4513-bd15-fdc6b866d151 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:37.116359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1066a-4e23-4513-bd15-fdc6b866d151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:37.116698 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1066a-4e23-4513-bd15-fdc6b866d151 admin admin] PATH_INFO: `/v3/projects/47f4599f9477451788ea1636cc3dc48b/users/4d4b56997201406bbe2cc13f72f8d06c/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:37.134348 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1066a-4e23-4513-bd15-fdc6b866d151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47f4599f9477451788ea1636cc3dc48b, user_id=4d4b56997201406bbe2cc13f72f8d06c, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:37.137029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1066a-4e23-4513-bd15-fdc6b866d151 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:37.155527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1075/2094] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:37 2022] PUT /identity/v3/projects/47f4599f9477451788ea1636cc3dc48b/users/4d4b56997201406bbe2cc13f72f8d06c/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:37.155755 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a0959-d075-41f0-9b6f-0ebb9ca3dd3c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:37.159208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1066a-4e23-4513-bd15-fdc6b866d151 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:37.161538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:37.216021 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:37.228971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9bd17-c862-48ca-bed8-1678efeee152 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:37.229915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9bd17-c862-48ca-bed8-1678efeee152 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:37.230434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9bd17-c862-48ca-bed8-1678efeee152 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:37.230714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9bd17-c862-48ca-bed8-1678efeee152 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:37.232850 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c2c9bd17-c862-48ca-bed8-1678efeee152 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:37.236333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1066a-4e23-4513-bd15-fdc6b866d151 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:37.242158 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd8e84-968b-4144-a76d-7d4d4a353856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:37.242913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd8e84-968b-4144-a76d-7d4d4a353856 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:37.243204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd8e84-968b-4144-a76d-7d4d4a353856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:37.243485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd8e84-968b-4144-a76d-7d4d4a353856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:37.244224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd8e84-968b-4144-a76d-7d4d4a353856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:37.245850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd8e84-968b-4144-a76d-7d4d4a353856 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:37.259682 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1076/2095] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:37.263560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd8e84-968b-4144-a76d-7d4d4a353856 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:37.275982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd8e84-968b-4144-a76d-7d4d4a353856 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:37.281095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32b8c6-ba98-4d5b-8ca7-96a387ebfba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:37.282017 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32b8c6-ba98-4d5b-8ca7-96a387ebfba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:37.282428 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32b8c6-ba98-4d5b-8ca7-96a387ebfba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:37.282716 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32b8c6-ba98-4d5b-8ca7-96a387ebfba1 admin admin] PATH_INFO: `/v3/projects/47f4599f9477451788ea1636cc3dc48b/users/4d4b56997201406bbe2cc13f72f8d06c/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:37.284811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32b8c6-ba98-4d5b-8ca7-96a387ebfba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47f4599f9477451788ea1636cc3dc48b, user_id=4d4b56997201406bbe2cc13f72f8d06c, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:37.287629 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32b8c6-ba98-4d5b-8ca7-96a387ebfba1 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:37.300190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1077/2096] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:37 2022] PUT /identity/v3/projects/47f4599f9477451788ea1636cc3dc48b/users/4d4b56997201406bbe2cc13f72f8d06c/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:43:37.319643 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fac9d-3b54-4ec6-a773-ed89906203f0 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:37.319965 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fac9d-3b54-4ec6-a773-ed89906203f0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:37.320216 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fac9d-3b54-4ec6-a773-ed89906203f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:37.338404 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9bd17-c862-48ca-bed8-1678efeee152 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:37.340047 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9bd17-c862-48ca-bed8-1678efeee152 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:37.352847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1020/2097] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:43:37.391560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9e1fac9d-3b54-4ec6-a773-ed89906203f0 None None] MFA Rules not processed for user `4d4b56997201406bbe2cc13f72f8d06c`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:37.393719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9e1fac9d-3b54-4ec6-a773-ed89906203f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:37.533652 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1078/2098] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:37 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:37.548647 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9bd17-c862-48ca-bed8-1678efeee152 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:37.550056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c2c9bd17-c862-48ca-bed8-1678efeee152 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:37.571758 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9bd17-c862-48ca-bed8-1678efeee152 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:37.669586 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008948e-de98-4a6e-abd8-0db068ea73ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:37.670319 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008948e-de98-4a6e-abd8-0db068ea73ea service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:37.671314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008948e-de98-4a6e-abd8-0db068ea73ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:37.671610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008948e-de98-4a6e-abd8-0db068ea73ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:37.692140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008948e-de98-4a6e-abd8-0db068ea73ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:37.693784 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008948e-de98-4a6e-abd8-0db068ea73ea service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:37.709021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1021/2099] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:37 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:37.837435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fac9d-3b54-4ec6-a773-ed89906203f0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:37.859683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fac9d-3b54-4ec6-a773-ed89906203f0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:37.872246 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c6874-b1a2-4ef2-b92d-bfa132ed81c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:37.873249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c6874-b1a2-4ef2-b92d-bfa132ed81c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:37.873872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c6874-b1a2-4ef2-b92d-bfa132ed81c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:37.874387 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c6874-b1a2-4ef2-b92d-bfa132ed81c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:37.988017 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c6874-b1a2-4ef2-b92d-bfa132ed81c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:37.992138 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c6874-b1a2-4ef2-b92d-bfa132ed81c9 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:38.023500 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1079/2100] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:42.265567 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008948e-de98-4a6e-abd8-0db068ea73ea service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:42.341999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008948e-de98-4a6e-abd8-0db068ea73ea service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:42.346624 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650852c-ca1c-4137-8adc-50bdd631d8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:42.348109 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650852c-ca1c-4137-8adc-50bdd631d8c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:42.349002 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650852c-ca1c-4137-8adc-50bdd631d8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:42.349395 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650852c-ca1c-4137-8adc-50bdd631d8c4 admin admin] PATH_INFO: `/v3/users/49188820bbeb44509a642ca4ba81fc2d` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:42.350735 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650852c-ca1c-4137-8adc-50bdd631d8c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49188820bbeb44509a642ca4ba81fc2d)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:42.353255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650852c-ca1c-4137-8adc-50bdd631d8c4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:42.356272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:42.404235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3650852c-ca1c-4137-8adc-50bdd631d8c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49188820bbeb44509a642ca4ba81fc2d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:42.417627 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3650852c-ca1c-4137-8adc-50bdd631d8c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49188820bbeb44509a642ca4ba81fc2d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:42.432856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3650852c-ca1c-4137-8adc-50bdd631d8c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49188820bbeb44509a642ca4ba81fc2d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:42.433618 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3650852c-ca1c-4137-8adc-50bdd631d8c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49188820bbeb44509a642ca4ba81fc2d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:42.442803 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3650852c-ca1c-4137-8adc-50bdd631d8c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49188820bbeb44509a642ca4ba81fc2d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:42.450257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1022/2101] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:42 2022] DELETE /identity/v3/users/49188820bbeb44509a642ca4ba81fc2d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:42.454486 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd8262-eaaa-4af3-919d-047be7e1b1a6 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:42.455009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd8262-eaaa-4af3-919d-047be7e1b1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:42.455121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd8262-eaaa-4af3-919d-047be7e1b1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:42.473477 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d2fd8262-eaaa-4af3-919d-047be7e1b1a6 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:42.474730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d2fd8262-eaaa-4af3-919d-047be7e1b1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:42.523821 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:42.537271 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1080/2102] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:43:42.547167 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650852c-ca1c-4137-8adc-50bdd631d8c4 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:42.548677 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3650852c-ca1c-4137-8adc-50bdd631d8c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:42.551774 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd8262-eaaa-4af3-919d-047be7e1b1a6 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:42.553213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d2fd8262-eaaa-4af3-919d-047be7e1b1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:42.566196 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd8262-eaaa-4af3-919d-047be7e1b1a6 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:42.571259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5a2ca-77af-414b-8e57-332bcece4bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:42.572135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5a2ca-77af-414b-8e57-332bcece4bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:42.572420 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5a2ca-77af-414b-8e57-332bcece4bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:42.572698 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5a2ca-77af-414b-8e57-332bcece4bd4 admin admin] PATH_INFO: `/v3/users/aba49623ff4749279e52d22821b07261` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:42.573902 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5a2ca-77af-414b-8e57-332bcece4bd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aba49623ff4749279e52d22821b07261)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:42.575871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5a2ca-77af-414b-8e57-332bcece4bd4 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:42.620845 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650852c-ca1c-4137-8adc-50bdd631d8c4 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:42.624825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403a334-5e01-4a63-af9c-f9a47b2387c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:42.625457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403a334-5e01-4a63-af9c-f9a47b2387c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:42.625743 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403a334-5e01-4a63-af9c-f9a47b2387c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:42.626053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403a334-5e01-4a63-af9c-f9a47b2387c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:42.633567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e5c5a2ca-77af-414b-8e57-332bcece4bd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aba49623ff4749279e52d22821b07261'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:42.634486 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e5c5a2ca-77af-414b-8e57-332bcece4bd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aba49623ff4749279e52d22821b07261'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:42.635127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e5c5a2ca-77af-414b-8e57-332bcece4bd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aba49623ff4749279e52d22821b07261'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:42.636678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2403a334-5e01-4a63-af9c-f9a47b2387c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:42.638257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2403a334-5e01-4a63-af9c-f9a47b2387c9 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:42.650982 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1023/2103] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:42.654170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e5c5a2ca-77af-414b-8e57-332bcece4bd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aba49623ff4749279e52d22821b07261'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:42.667114 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e5c5a2ca-77af-414b-8e57-332bcece4bd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aba49623ff4749279e52d22821b07261'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:42.684479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1081/2104] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:42 2022] DELETE /identity/v3/users/aba49623ff4749279e52d22821b07261 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:43:42.689109 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637e43c-71ea-4b96-ad7d-5e90733a86ad None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:42.689621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637e43c-71ea-4b96-ad7d-5e90733a86ad None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:42.689918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637e43c-71ea-4b96-ad7d-5e90733a86ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:42.713297 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-5637e43c-71ea-4b96-ad7d-5e90733a86ad None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:42.714732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5637e43c-71ea-4b96-ad7d-5e90733a86ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:42.806736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1024/2105] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:42.816007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5a2ca-77af-414b-8e57-332bcece4bd4 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:42.817945 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e5c5a2ca-77af-414b-8e57-332bcece4bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:42.906412 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5a2ca-77af-414b-8e57-332bcece4bd4 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:42.911350 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc0ac0-b068-46ac-9a8c-ff66b0187600 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:42.912017 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc0ac0-b068-46ac-9a8c-ff66b0187600 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:42.912328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc0ac0-b068-46ac-9a8c-ff66b0187600 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:42.912614 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc0ac0-b068-46ac-9a8c-ff66b0187600 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:42.922674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc0ac0-b068-46ac-9a8c-ff66b0187600 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:42.924137 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc0ac0-b068-46ac-9a8c-ff66b0187600 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:42.934399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1082/2106] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:42.981010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637e43c-71ea-4b96-ad7d-5e90733a86ad None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:42.982613 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5637e43c-71ea-4b96-ad7d-5e90733a86ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:43.002534 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637e43c-71ea-4b96-ad7d-5e90733a86ad None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:43.008632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6b308-6abc-4990-b171-6f750795329f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:43.009849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6b308-6abc-4990-b171-6f750795329f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:43.011021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6b308-6abc-4990-b171-6f750795329f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:43.011396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6b308-6abc-4990-b171-6f750795329f admin admin] PATH_INFO: `/v3/projects/f8f8b0241d1f48b6b929be821b75765f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:43.013757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6b308-6abc-4990-b171-6f750795329f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8f8b0241d1f48b6b929be821b75765f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:43.016796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6b308-6abc-4990-b171-6f750795329f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:43.051929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:43.080466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6ef6b308-6abc-4990-b171-6f750795329f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8f8b0241d1f48b6b929be821b75765f', 'request_id': 'req-6ef6b308-6abc-4990-b171-6f750795329f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.086855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6ef6b308-6abc-4990-b171-6f750795329f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8f8b0241d1f48b6b929be821b75765f', 'request_id': 'req-6ef6b308-6abc-4990-b171-6f750795329f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.097337 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6ef6b308-6abc-4990-b171-6f750795329f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8f8b0241d1f48b6b929be821b75765f', 'request_id': 'req-6ef6b308-6abc-4990-b171-6f750795329f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.111394 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6ef6b308-6abc-4990-b171-6f750795329f admin admin] The token cache is being invalidate because project f8f8b0241d1f48b6b929be821b75765f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:43.111693 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6ef6b308-6abc-4990-b171-6f750795329f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.112371 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6ef6b308-6abc-4990-b171-6f750795329f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.114357 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1025/2107] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:42 2022] DELETE /identity/v3/projects/f8f8b0241d1f48b6b929be821b75765f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:43.118317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc0ac0-b068-46ac-9a8c-ff66b0187600 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:43.129462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc0ac0-b068-46ac-9a8c-ff66b0187600 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:43.198168 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b8792-0bea-48ef-b1d9-e28d0ac30e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:43.200340 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b8792-0bea-48ef-b1d9-e28d0ac30e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:43.200843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b8792-0bea-48ef-b1d9-e28d0ac30e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:43.201353 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b8792-0bea-48ef-b1d9-e28d0ac30e3c admin admin] PATH_INFO: `/v3/users/4d4b56997201406bbe2cc13f72f8d06c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:43.202983 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b8792-0bea-48ef-b1d9-e28d0ac30e3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d4b56997201406bbe2cc13f72f8d06c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:43.205261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b8792-0bea-48ef-b1d9-e28d0ac30e3c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:43.223838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:43.234925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6b308-6abc-4990-b171-6f750795329f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:43.236623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6ef6b308-6abc-4990-b171-6f750795329f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:43.258428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6b308-6abc-4990-b171-6f750795329f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:43.265516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83e574-e827-424d-8b5b-1feaf04eec6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:43.266661 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83e574-e827-424d-8b5b-1feaf04eec6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:43.267205 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83e574-e827-424d-8b5b-1feaf04eec6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:43.267394 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83e574-e827-424d-8b5b-1feaf04eec6c admin admin] PATH_INFO: `/v3/projects/d42e7529420f4b69a691768ec0ca1b25` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:43.268701 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83e574-e827-424d-8b5b-1feaf04eec6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d42e7529420f4b69a691768ec0ca1b25)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:43.271457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83e574-e827-424d-8b5b-1feaf04eec6c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:43.303228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-635b8792-0bea-48ef-b1d9-e28d0ac30e3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d4b56997201406bbe2cc13f72f8d06c'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.303890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-635b8792-0bea-48ef-b1d9-e28d0ac30e3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d4b56997201406bbe2cc13f72f8d06c'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.304506 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-635b8792-0bea-48ef-b1d9-e28d0ac30e3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d4b56997201406bbe2cc13f72f8d06c'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.322521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-635b8792-0bea-48ef-b1d9-e28d0ac30e3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d4b56997201406bbe2cc13f72f8d06c'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.329813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ff83e574-e827-424d-8b5b-1feaf04eec6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd42e7529420f4b69a691768ec0ca1b25', 'request_id': 'req-ff83e574-e827-424d-8b5b-1feaf04eec6c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.330670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-635b8792-0bea-48ef-b1d9-e28d0ac30e3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d4b56997201406bbe2cc13f72f8d06c'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.337944 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ff83e574-e827-424d-8b5b-1feaf04eec6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd42e7529420f4b69a691768ec0ca1b25', 'request_id': 'req-ff83e574-e827-424d-8b5b-1feaf04eec6c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.346842 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ff83e574-e827-424d-8b5b-1feaf04eec6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd42e7529420f4b69a691768ec0ca1b25', 'request_id': 'req-ff83e574-e827-424d-8b5b-1feaf04eec6c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.348224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1083/2108] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:43 2022] DELETE /identity/v3/users/4d4b56997201406bbe2cc13f72f8d06c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:43.379320 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ff83e574-e827-424d-8b5b-1feaf04eec6c admin admin] The token cache is being invalidate because project d42e7529420f4b69a691768ec0ca1b25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:43.379686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ff83e574-e827-424d-8b5b-1feaf04eec6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.380418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ff83e574-e827-424d-8b5b-1feaf04eec6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.382330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1026/2109] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:43 2022] DELETE /identity/v3/projects/d42e7529420f4b69a691768ec0ca1b25 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:43.386095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b8792-0bea-48ef-b1d9-e28d0ac30e3c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:43.387565 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-635b8792-0bea-48ef-b1d9-e28d0ac30e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:43.454015 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b8792-0bea-48ef-b1d9-e28d0ac30e3c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:43.459916 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488e348-4701-438b-b1b2-57fe81b7bcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:43.460867 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488e348-4701-438b-b1b2-57fe81b7bcab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:43.461156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488e348-4701-438b-b1b2-57fe81b7bcab admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:43.461440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488e348-4701-438b-b1b2-57fe81b7bcab admin admin] PATH_INFO: `/v3/users/13b82548e9314119af0eab8733d238f7` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:43.462718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488e348-4701-438b-b1b2-57fe81b7bcab admin admin] RBAC: Authorizing `identity:delete_user(user_id=13b82548e9314119af0eab8733d238f7)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:43.464835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488e348-4701-438b-b1b2-57fe81b7bcab admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:43.521505 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3488e348-4701-438b-b1b2-57fe81b7bcab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13b82548e9314119af0eab8733d238f7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.522619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3488e348-4701-438b-b1b2-57fe81b7bcab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13b82548e9314119af0eab8733d238f7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.523635 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3488e348-4701-438b-b1b2-57fe81b7bcab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13b82548e9314119af0eab8733d238f7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.546397 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3488e348-4701-438b-b1b2-57fe81b7bcab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13b82548e9314119af0eab8733d238f7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.554311 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3488e348-4701-438b-b1b2-57fe81b7bcab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13b82548e9314119af0eab8733d238f7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.568204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1084/2110] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:43 2022] DELETE /identity/v3/users/13b82548e9314119af0eab8733d238f7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:43.658611 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83e574-e827-424d-8b5b-1feaf04eec6c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:43.660167 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ff83e574-e827-424d-8b5b-1feaf04eec6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:43.761514 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83e574-e827-424d-8b5b-1feaf04eec6c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:43.795538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:43.821553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659e8a5-51a0-4fb5-a7ce-4d7761448d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:43.822439 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659e8a5-51a0-4fb5-a7ce-4d7761448d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:43.822727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659e8a5-51a0-4fb5-a7ce-4d7761448d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:43.823009 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659e8a5-51a0-4fb5-a7ce-4d7761448d32 admin admin] PATH_INFO: `/v3/projects/47f4599f9477451788ea1636cc3dc48b` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:43.824203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659e8a5-51a0-4fb5-a7ce-4d7761448d32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47f4599f9477451788ea1636cc3dc48b)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:43.826153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659e8a5-51a0-4fb5-a7ce-4d7761448d32 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:43.842501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488e348-4701-438b-b1b2-57fe81b7bcab admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:43.851415 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3488e348-4701-438b-b1b2-57fe81b7bcab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:43.879593 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488e348-4701-438b-b1b2-57fe81b7bcab admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:43.889224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed91a54-a002-4da5-a6de-9c0e5b947898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:43.890581 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed91a54-a002-4da5-a6de-9c0e5b947898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:43.890951 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed91a54-a002-4da5-a6de-9c0e5b947898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:43.891260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed91a54-a002-4da5-a6de-9c0e5b947898 admin admin] PATH_INFO: `/v3/projects/21ca3b9964584e7a8d88df373d824038` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:43.892601 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed91a54-a002-4da5-a6de-9c0e5b947898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21ca3b9964584e7a8d88df373d824038)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:43.894844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed91a54-a002-4da5-a6de-9c0e5b947898 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:43.898016 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:43.916449 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c659e8a5-51a0-4fb5-a7ce-4d7761448d32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47f4599f9477451788ea1636cc3dc48b', 'request_id': 'req-c659e8a5-51a0-4fb5-a7ce-4d7761448d32'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.923027 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c659e8a5-51a0-4fb5-a7ce-4d7761448d32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47f4599f9477451788ea1636cc3dc48b', 'request_id': 'req-c659e8a5-51a0-4fb5-a7ce-4d7761448d32'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.931623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c659e8a5-51a0-4fb5-a7ce-4d7761448d32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47f4599f9477451788ea1636cc3dc48b', 'request_id': 'req-c659e8a5-51a0-4fb5-a7ce-4d7761448d32'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.950985 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c659e8a5-51a0-4fb5-a7ce-4d7761448d32 admin admin] The token cache is being invalidate because project 47f4599f9477451788ea1636cc3dc48b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:43.951289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c659e8a5-51a0-4fb5-a7ce-4d7761448d32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.952382 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c659e8a5-51a0-4fb5-a7ce-4d7761448d32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.954465 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1027/2111] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:43 2022] DELETE /identity/v3/projects/47f4599f9477451788ea1636cc3dc48b => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:43.958277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7ed91a54-a002-4da5-a6de-9c0e5b947898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21ca3b9964584e7a8d88df373d824038', 'request_id': 'req-7ed91a54-a002-4da5-a6de-9c0e5b947898'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.959635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d616366-ce61-489c-815c-fe209fb1a686 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:43.960143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d616366-ce61-489c-815c-fe209fb1a686 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:43.960567 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d616366-ce61-489c-815c-fe209fb1a686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:43.967588 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7ed91a54-a002-4da5-a6de-9c0e5b947898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21ca3b9964584e7a8d88df373d824038', 'request_id': 'req-7ed91a54-a002-4da5-a6de-9c0e5b947898'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.977545 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7ed91a54-a002-4da5-a6de-9c0e5b947898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21ca3b9964584e7a8d88df373d824038', 'request_id': 'req-7ed91a54-a002-4da5-a6de-9c0e5b947898'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:43.994426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3d616366-ce61-489c-815c-fe209fb1a686 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:43.995771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3d616366-ce61-489c-815c-fe209fb1a686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:44.002625 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7ed91a54-a002-4da5-a6de-9c0e5b947898 admin admin] The token cache is being invalidate because project 21ca3b9964584e7a8d88df373d824038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:44.003722 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7ed91a54-a002-4da5-a6de-9c0e5b947898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:44.004444 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7ed91a54-a002-4da5-a6de-9c0e5b947898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:44.006667 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1085/2112] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:43 2022] DELETE /identity/v3/projects/21ca3b9964584e7a8d88df373d824038 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:44.012710 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438a0d7-45e6-45f9-a2d3-228d53439bee None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.012993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438a0d7-45e6-45f9-a2d3-228d53439bee None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.013332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438a0d7-45e6-45f9-a2d3-228d53439bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.033625 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3438a0d7-45e6-45f9-a2d3-228d53439bee None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:44.035456 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3438a0d7-45e6-45f9-a2d3-228d53439bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:44.067545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1028/2113] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:43:44.075262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b4933-68a1-4f79-b9dc-7d511c4438cf None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.075495 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b4933-68a1-4f79-b9dc-7d511c4438cf None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.075780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b4933-68a1-4f79-b9dc-7d511c4438cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.092957 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-cd7b4933-68a1-4f79-b9dc-7d511c4438cf None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:44.094239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cd7b4933-68a1-4f79-b9dc-7d511c4438cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:44.099253 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1086/2114] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:43:44.102976 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1029/2115] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:43:44.106776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b4933-68a1-4f79-b9dc-7d511c4438cf None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:44.107071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baee36-c1a3-446d-bb57-9e5e7c4bf02d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.107432 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baee36-c1a3-446d-bb57-9e5e7c4bf02d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.107778 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baee36-c1a3-446d-bb57-9e5e7c4bf02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.116650 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b4933-68a1-4f79-b9dc-7d511c4438cf None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:44.123824 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf743f1-28f0-4f38-868a-3586ed1a4bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:44.123824 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf743f1-28f0-4f38-868a-3586ed1a4bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.123824 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf743f1-28f0-4f38-868a-3586ed1a4bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.123824 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf743f1-28f0-4f38-868a-3586ed1a4bf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.124242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf743f1-28f0-4f38-868a-3586ed1a4bf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:43:44.124242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf743f1-28f0-4f38-868a-3586ed1a4bf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:44.125360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf743f1-28f0-4f38-868a-3586ed1a4bf5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:44.128550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-13baee36-c1a3-446d-bb57-9e5e7c4bf02d None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:44.129858 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-13baee36-c1a3-446d-bb57-9e5e7c4bf02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:44.142106 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1030/2116] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:43:44.143754 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1087/2117] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:44.146316 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf743f1-28f0-4f38-868a-3586ed1a4bf5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:44.147723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13baee36-c1a3-446d-bb57-9e5e7c4bf02d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:44.156235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf743f1-28f0-4f38-868a-3586ed1a4bf5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:44.156678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13baee36-c1a3-446d-bb57-9e5e7c4bf02d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:44.161551 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78712e1c-8c18-4ddc-8c3a-aa448068ab4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:44.162007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75037d-523a-4fd6-9139-d9484b7c9c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:44.162235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78712e1c-8c18-4ddc-8c3a-aa448068ab4e admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.162523 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78712e1c-8c18-4ddc-8c3a-aa448068ab4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.162651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75037d-523a-4fd6-9139-d9484b7c9c5b admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.162799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78712e1c-8c18-4ddc-8c3a-aa448068ab4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.162939 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75037d-523a-4fd6-9139-d9484b7c9c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.163250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75037d-523a-4fd6-9139-d9484b7c9c5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.163484 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78712e1c-8c18-4ddc-8c3a-aa448068ab4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:44.163970 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75037d-523a-4fd6-9139-d9484b7c9c5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:43:44.164280 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75037d-523a-4fd6-9139-d9484b7c9c5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:44.165411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78712e1c-8c18-4ddc-8c3a-aa448068ab4e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:44.165883 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75037d-523a-4fd6-9139-d9484b7c9c5b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:44.178788 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1088/2118] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:43:44.183272 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75037d-523a-4fd6-9139-d9484b7c9c5b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:44.192328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1031/2119] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:44 2022] POST /identity/v3/projects => generated 359 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:43:44.193300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75037d-523a-4fd6-9139-d9484b7c9c5b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:44.196864 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78712e1c-8c18-4ddc-8c3a-aa448068ab4e admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:44.227663 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:44.255324 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a0b02-0748-4b91-b28d-850ab9773093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:44.255985 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a0b02-0748-4b91-b28d-850ab9773093 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.256410 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a0b02-0748-4b91-b28d-850ab9773093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.256705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a0b02-0748-4b91-b28d-850ab9773093 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.257372 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a0b02-0748-4b91-b28d-850ab9773093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:44.259377 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a0b02-0748-4b91-b28d-850ab9773093 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:44.262507 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78712e1c-8c18-4ddc-8c3a-aa448068ab4e admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:44.268125 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f218af-4ef6-4d24-baf6-611f7b968668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:44.269114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f218af-4ef6-4d24-baf6-611f7b968668 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.271341 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f218af-4ef6-4d24-baf6-611f7b968668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.271341 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f218af-4ef6-4d24-baf6-611f7b968668 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.271341 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f218af-4ef6-4d24-baf6-611f7b968668 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:44.274438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f218af-4ef6-4d24-baf6-611f7b968668 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:44.289296 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:44.302801 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1089/2120] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:44 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:43:44.303705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1032/2121] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:44 2022] POST /identity/v3/users => generated 280 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:43:44.307931 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a0b02-0748-4b91-b28d-850ab9773093 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:44.308101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f218af-4ef6-4d24-baf6-611f7b968668 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:44.321122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f218af-4ef6-4d24-baf6-611f7b968668 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:44.378298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a0b02-0748-4b91-b28d-850ab9773093 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:44.387595 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e04d4-34f2-455d-8b5b-97e20cf54fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:44.388283 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e04d4-34f2-455d-8b5b-97e20cf54fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.388566 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e04d4-34f2-455d-8b5b-97e20cf54fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.388853 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e04d4-34f2-455d-8b5b-97e20cf54fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.389571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e04d4-34f2-455d-8b5b-97e20cf54fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:44.391366 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e04d4-34f2-455d-8b5b-97e20cf54fe1 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:44.396721 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46d4c2-6a0a-40c1-b682-61ced6715c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:44.397589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46d4c2-6a0a-40c1-b682-61ced6715c15 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.397886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46d4c2-6a0a-40c1-b682-61ced6715c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.398609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46d4c2-6a0a-40c1-b682-61ced6715c15 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.399622 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46d4c2-6a0a-40c1-b682-61ced6715c15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:44.401879 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46d4c2-6a0a-40c1-b682-61ced6715c15 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:44.403603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1090/2122] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:44.407656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e04d4-34f2-455d-8b5b-97e20cf54fe1 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:44.421192 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e04d4-34f2-455d-8b5b-97e20cf54fe1 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:44.426886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32355443-0e5c-4c58-a896-93536e80f2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:44.427839 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32355443-0e5c-4c58-a896-93536e80f2b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.428253 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32355443-0e5c-4c58-a896-93536e80f2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.428625 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32355443-0e5c-4c58-a896-93536e80f2b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.429537 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32355443-0e5c-4c58-a896-93536e80f2b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:44.430752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1033/2123] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:44 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:44.431515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32355443-0e5c-4c58-a896-93536e80f2b5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:44.434978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46d4c2-6a0a-40c1-b682-61ced6715c15 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:44.449222 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1091/2124] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:43:44.450420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46d4c2-6a0a-40c1-b682-61ced6715c15 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:44.452505 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32355443-0e5c-4c58-a896-93536e80f2b5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:44.457320 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fb8f8-c785-4c20-bff3-6027a36f6fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:44.458231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fb8f8-c785-4c20-bff3-6027a36f6fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.458683 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fb8f8-c785-4c20-bff3-6027a36f6fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.459113 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fb8f8-c785-4c20-bff3-6027a36f6fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.460220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797fb8f8-c785-4c20-bff3-6027a36f6fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:44.462601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797fb8f8-c785-4c20-bff3-6027a36f6fa6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:44.473662 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32355443-0e5c-4c58-a896-93536e80f2b5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:44.475543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1034/2125] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:43:44.479691 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fb8f8-c785-4c20-bff3-6027a36f6fa6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:44.480007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d937562-1daf-4ba3-abc4-d73d86947953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:44.481804 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d937562-1daf-4ba3-abc4-d73d86947953 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.482605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d937562-1daf-4ba3-abc4-d73d86947953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.483096 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d937562-1daf-4ba3-abc4-d73d86947953 admin admin] PATH_INFO: `/v3/projects/2b6de89a7a42465ea4db09e7b288afa8/users/1e1f9a11e6ce41d0b9c7649e9623f062/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.500324 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fb8f8-c785-4c20-bff3-6027a36f6fa6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:44.505162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d937562-1daf-4ba3-abc4-d73d86947953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b6de89a7a42465ea4db09e7b288afa8, user_id=1e1f9a11e6ce41d0b9c7649e9623f062, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:44.508026 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d937562-1daf-4ba3-abc4-d73d86947953 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:44.510000 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb5e2e-31c6-4b9f-934b-d6196d27f4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:44.511253 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb5e2e-31c6-4b9f-934b-d6196d27f4dc admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.512073 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb5e2e-31c6-4b9f-934b-d6196d27f4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.512438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb5e2e-31c6-4b9f-934b-d6196d27f4dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.513691 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb5e2e-31c6-4b9f-934b-d6196d27f4dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:44.516467 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb5e2e-31c6-4b9f-934b-d6196d27f4dc admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:44.537472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1035/2126] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:44.549431 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb5e2e-31c6-4b9f-934b-d6196d27f4dc admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:44.549647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1092/2127] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:44 2022] PUT /identity/v3/projects/2b6de89a7a42465ea4db09e7b288afa8/users/1e1f9a11e6ce41d0b9c7649e9623f062/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:44.551448 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcccef6-5918-4629-a3f5-5712e7858806 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.552660 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcccef6-5918-4629-a3f5-5712e7858806 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.552949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcccef6-5918-4629-a3f5-5712e7858806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.569879 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb5e2e-31c6-4b9f-934b-d6196d27f4dc admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:44.596471 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-abcccef6-5918-4629-a3f5-5712e7858806 None None] MFA Rules not processed for user `1e1f9a11e6ce41d0b9c7649e9623f062`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:44.598453 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-abcccef6-5918-4629-a3f5-5712e7858806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:44.670905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464673d-b01b-4aaa-b82f-bdbccb7d4e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:44.671905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464673d-b01b-4aaa-b82f-bdbccb7d4e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.672246 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464673d-b01b-4aaa-b82f-bdbccb7d4e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.672584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464673d-b01b-4aaa-b82f-bdbccb7d4e94 admin admin] PATH_INFO: `/v3/projects/3b8cc00fdb9343d7816dd5a171dd518b/users/b650f86088e74077b37a5ae178f90ba0/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.687338 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1093/2128] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:44 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:44.689911 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464673d-b01b-4aaa-b82f-bdbccb7d4e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b8cc00fdb9343d7816dd5a171dd518b, user_id=b650f86088e74077b37a5ae178f90ba0, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:44.693846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464673d-b01b-4aaa-b82f-bdbccb7d4e94 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:44.710422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1036/2129] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:44 2022] PUT /identity/v3/projects/3b8cc00fdb9343d7816dd5a171dd518b/users/b650f86088e74077b37a5ae178f90ba0/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:44.721363 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcccef6-5918-4629-a3f5-5712e7858806 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:44.723011 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-abcccef6-5918-4629-a3f5-5712e7858806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:44.789459 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5494e-b8d1-49d4-a1f0-f7daa79fcb04 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.789859 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5494e-b8d1-49d4-a1f0-f7daa79fcb04 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.790180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5494e-b8d1-49d4-a1f0-f7daa79fcb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.799252 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:44.805490 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcccef6-5918-4629-a3f5-5712e7858806 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:44.808835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:44.827033 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d3c5494e-b8d1-49d4-a1f0-f7daa79fcb04 None None] MFA Rules not processed for user `b650f86088e74077b37a5ae178f90ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:44.828340 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d3c5494e-b8d1-49d4-a1f0-f7daa79fcb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:44.862951 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014d1cd-c1f6-4e6c-ad6d-2d5214e9bf76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:44.863662 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014d1cd-c1f6-4e6c-ad6d-2d5214e9bf76 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.863973 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014d1cd-c1f6-4e6c-ad6d-2d5214e9bf76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.864287 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014d1cd-c1f6-4e6c-ad6d-2d5214e9bf76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.905216 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1037/2130] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:44.917710 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5494e-b8d1-49d4-a1f0-f7daa79fcb04 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:44.919262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d3c5494e-b8d1-49d4-a1f0-f7daa79fcb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:44.930551 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014d1cd-c1f6-4e6c-ad6d-2d5214e9bf76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:44.931804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5494e-b8d1-49d4-a1f0-f7daa79fcb04 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:44.932024 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014d1cd-c1f6-4e6c-ad6d-2d5214e9bf76 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:44.935496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d1e49-60c7-4d48-88bb-e1b7f2cbd1ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:44.936132 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d1e49-60c7-4d48-88bb-e1b7f2cbd1ba service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:44.936409 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d1e49-60c7-4d48-88bb-e1b7f2cbd1ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:44.936679 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d1e49-60c7-4d48-88bb-e1b7f2cbd1ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:44.942596 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1094/2131] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:44 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:44.946067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d1e49-60c7-4d48-88bb-e1b7f2cbd1ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:44.947488 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d1e49-60c7-4d48-88bb-e1b7f2cbd1ba service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:44.956253 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014d1cd-c1f6-4e6c-ad6d-2d5214e9bf76 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:44.960423 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1038/2132] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:43:45.025583 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014d1cd-c1f6-4e6c-ad6d-2d5214e9bf76 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:45.030408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb82ea-f5f2-4682-ba92-d181779f14b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:45.031289 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb82ea-f5f2-4682-ba92-d181779f14b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:45.031610 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb82ea-f5f2-4682-ba92-d181779f14b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:45.031888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb82ea-f5f2-4682-ba92-d181779f14b1 admin admin] PATH_INFO: `/v3/users/1e1f9a11e6ce41d0b9c7649e9623f062` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:45.033095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb82ea-f5f2-4682-ba92-d181779f14b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e1f9a11e6ce41d0b9c7649e9623f062)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:45.035222 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb82ea-f5f2-4682-ba92-d181779f14b1 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:45.088630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f1cb82ea-f5f2-4682-ba92-d181779f14b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e1f9a11e6ce41d0b9c7649e9623f062'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:45.089374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f1cb82ea-f5f2-4682-ba92-d181779f14b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e1f9a11e6ce41d0b9c7649e9623f062'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:45.090057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f1cb82ea-f5f2-4682-ba92-d181779f14b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e1f9a11e6ce41d0b9c7649e9623f062'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:45.108561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:45.111769 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f1cb82ea-f5f2-4682-ba92-d181779f14b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e1f9a11e6ce41d0b9c7649e9623f062'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:45.123130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f1cb82ea-f5f2-4682-ba92-d181779f14b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e1f9a11e6ce41d0b9c7649e9623f062'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:45.141991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1095/2133] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:44 2022] DELETE /identity/v3/users/1e1f9a11e6ce41d0b9c7649e9623f062 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:43:45.149013 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78609e1-b185-4716-ba12-644784fe6674 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:45.154016 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78609e1-b185-4716-ba12-644784fe6674 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:45.154016 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78609e1-b185-4716-ba12-644784fe6674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:45.176284 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d78609e1-b185-4716-ba12-644784fe6674 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:45.177646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d78609e1-b185-4716-ba12-644784fe6674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:45.246557 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1039/2134] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:45.261356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb82ea-f5f2-4682-ba92-d181779f14b1 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:45.262867 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f1cb82ea-f5f2-4682-ba92-d181779f14b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:45.357035 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb82ea-f5f2-4682-ba92-d181779f14b1 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:45.369387 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ae3c1-65a0-417a-91c4-b59b4cbc90f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:45.370082 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ae3c1-65a0-417a-91c4-b59b4cbc90f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:45.370371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ae3c1-65a0-417a-91c4-b59b4cbc90f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:45.370650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ae3c1-65a0-417a-91c4-b59b4cbc90f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:45.382644 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ae3c1-65a0-417a-91c4-b59b4cbc90f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:45.384329 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ae3c1-65a0-417a-91c4-b59b4cbc90f2 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:45.395362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1096/2135] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:45.678004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78609e1-b185-4716-ba12-644784fe6674 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:45.679531 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d78609e1-b185-4716-ba12-644784fe6674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:45.693166 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78609e1-b185-4716-ba12-644784fe6674 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:45.699018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa08959-e870-46b8-bbae-fc180dc5e921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:45.700067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa08959-e870-46b8-bbae-fc180dc5e921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:45.700427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa08959-e870-46b8-bbae-fc180dc5e921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:45.700778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa08959-e870-46b8-bbae-fc180dc5e921 admin admin] PATH_INFO: `/v3/projects/2b6de89a7a42465ea4db09e7b288afa8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:45.702154 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa08959-e870-46b8-bbae-fc180dc5e921 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b6de89a7a42465ea4db09e7b288afa8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:45.704217 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa08959-e870-46b8-bbae-fc180dc5e921 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:45.755369 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ffa08959-e870-46b8-bbae-fc180dc5e921 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b6de89a7a42465ea4db09e7b288afa8', 'request_id': 'req-ffa08959-e870-46b8-bbae-fc180dc5e921'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:45.760923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ffa08959-e870-46b8-bbae-fc180dc5e921 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b6de89a7a42465ea4db09e7b288afa8', 'request_id': 'req-ffa08959-e870-46b8-bbae-fc180dc5e921'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:45.768250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ffa08959-e870-46b8-bbae-fc180dc5e921 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b6de89a7a42465ea4db09e7b288afa8', 'request_id': 'req-ffa08959-e870-46b8-bbae-fc180dc5e921'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:45.779067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ffa08959-e870-46b8-bbae-fc180dc5e921 admin admin] The token cache is being invalidate because project 2b6de89a7a42465ea4db09e7b288afa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:45.779354 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ffa08959-e870-46b8-bbae-fc180dc5e921 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:45.779981 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ffa08959-e870-46b8-bbae-fc180dc5e921 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:45.781517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1040/2136] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:45 2022] DELETE /identity/v3/projects/2b6de89a7a42465ea4db09e7b288afa8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:45.787273 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bce66-42f5-477f-af4a-d9f34d979fc1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:45.787794 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bce66-42f5-477f-af4a-d9f34d979fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:45.787908 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bce66-42f5-477f-af4a-d9f34d979fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:45.807006 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3d9bce66-42f5-477f-af4a-d9f34d979fc1 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:45.808345 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3d9bce66-42f5-477f-af4a-d9f34d979fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:45.868487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1097/2137] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:45.877790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb8011-b290-4060-ae73-1ea51c147267 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:45.878061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb8011-b290-4060-ae73-1ea51c147267 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:45.878342 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb8011-b290-4060-ae73-1ea51c147267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:45.891210 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:45.898617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-cadb8011-b290-4060-ae73-1ea51c147267 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:45.900422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cadb8011-b290-4060-ae73-1ea51c147267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:45.908658 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1041/2138] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:45.912788 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bce66-42f5-477f-af4a-d9f34d979fc1 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:45.922595 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bce66-42f5-477f-af4a-d9f34d979fc1 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:45.927681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3b872-7c20-450c-aea9-25c64ec8faeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:45.928329 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3b872-7c20-450c-aea9-25c64ec8faeb admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:45.928616 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3b872-7c20-450c-aea9-25c64ec8faeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:45.928897 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3b872-7c20-450c-aea9-25c64ec8faeb admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:45.929621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff3b872-7c20-450c-aea9-25c64ec8faeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:43:45.929938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff3b872-7c20-450c-aea9-25c64ec8faeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:45.931589 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff3b872-7c20-450c-aea9-25c64ec8faeb admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:45.943379 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1098/2139] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:43:45.947484 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb8011-b290-4060-ae73-1ea51c147267 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:45.956787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb8011-b290-4060-ae73-1ea51c147267 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:45.963868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70be16-b9c2-4599-9e00-3e250bf57c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:45.964538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70be16-b9c2-4599-9e00-3e250bf57c4d admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:45.964817 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70be16-b9c2-4599-9e00-3e250bf57c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:45.965093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70be16-b9c2-4599-9e00-3e250bf57c4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:45.965786 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70be16-b9c2-4599-9e00-3e250bf57c4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:45.967707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70be16-b9c2-4599-9e00-3e250bf57c4d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:45.995509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1042/2140] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:45 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:45.996564 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3b872-7c20-450c-aea9-25c64ec8faeb admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:46.007267 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3b872-7c20-450c-aea9-25c64ec8faeb admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:46.068962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4c7a0-a062-44d7-94f4-2562a94e22ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:46.069631 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4c7a0-a062-44d7-94f4-2562a94e22ec admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:46.069919 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4c7a0-a062-44d7-94f4-2562a94e22ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:46.070230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4c7a0-a062-44d7-94f4-2562a94e22ec admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:46.070914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4c7a0-a062-44d7-94f4-2562a94e22ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:46.072853 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4c7a0-a062-44d7-94f4-2562a94e22ec admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:46.088387 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:46.098152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1099/2141] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:45 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:43:46.099749 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70be16-b9c2-4599-9e00-3e250bf57c4d admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:46.110163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70be16-b9c2-4599-9e00-3e250bf57c4d admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:46.115398 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2af2b7-0009-4094-9ad0-43bc3515159f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:46.116047 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2af2b7-0009-4094-9ad0-43bc3515159f admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:46.116328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2af2b7-0009-4094-9ad0-43bc3515159f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:46.116606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2af2b7-0009-4094-9ad0-43bc3515159f admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:46.117370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2af2b7-0009-4094-9ad0-43bc3515159f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:46.119006 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2af2b7-0009-4094-9ad0-43bc3515159f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:46.129760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1043/2142] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:46.133673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4c7a0-a062-44d7-94f4-2562a94e22ec admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:46.143648 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4c7a0-a062-44d7-94f4-2562a94e22ec admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:46.148318 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ebfec-4549-4e7d-bad1-8b87792d3b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:46.148966 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ebfec-4549-4e7d-bad1-8b87792d3b98 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:46.149251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ebfec-4549-4e7d-bad1-8b87792d3b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:46.149528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ebfec-4549-4e7d-bad1-8b87792d3b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:46.150293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ebfec-4549-4e7d-bad1-8b87792d3b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:46.151888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ebfec-4549-4e7d-bad1-8b87792d3b98 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:46.162105 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1100/2143] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:46.165883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2af2b7-0009-4094-9ad0-43bc3515159f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:46.176588 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2af2b7-0009-4094-9ad0-43bc3515159f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:46.181255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3c898-6a36-4850-a72f-d81d0a025610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:46.182226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3c898-6a36-4850-a72f-d81d0a025610 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:46.182507 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3c898-6a36-4850-a72f-d81d0a025610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:46.182778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3c898-6a36-4850-a72f-d81d0a025610 admin admin] PATH_INFO: `/v3/projects/56e816c04dd44f02bb7a56a3259ba82b/users/d46b939637e045a4b694606a601aea97/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:46.198286 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3c898-6a36-4850-a72f-d81d0a025610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56e816c04dd44f02bb7a56a3259ba82b, user_id=d46b939637e045a4b694606a601aea97, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:46.200940 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3c898-6a36-4850-a72f-d81d0a025610 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:46.214571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1044/2144] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:46 2022] PUT /identity/v3/projects/56e816c04dd44f02bb7a56a3259ba82b/users/d46b939637e045a4b694606a601aea97/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:43:46.225357 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8088bd-4981-4fcb-94eb-0b263866bb06 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:46.225648 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8088bd-4981-4fcb-94eb-0b263866bb06 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:46.225927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8088bd-4981-4fcb-94eb-0b263866bb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:46.252644 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-cc8088bd-4981-4fcb-94eb-0b263866bb06 None None] MFA Rules not processed for user `d46b939637e045a4b694606a601aea97`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:46.253871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cc8088bd-4981-4fcb-94eb-0b263866bb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:46.318057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1101/2145] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:46.329034 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3c898-6a36-4850-a72f-d81d0a025610 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:46.330960 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0bc3c898-6a36-4850-a72f-d81d0a025610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:46.402214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3c898-6a36-4850-a72f-d81d0a025610 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:46.406457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac254c-8688-4775-8422-bdff718fe4c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:46.407121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac254c-8688-4775-8422-bdff718fe4c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:46.407412 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac254c-8688-4775-8422-bdff718fe4c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:46.407697 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac254c-8688-4775-8422-bdff718fe4c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:46.417254 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac254c-8688-4775-8422-bdff718fe4c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:46.419420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac254c-8688-4775-8422-bdff718fe4c9 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:46.429584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1045/2146] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:55.348319 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8088bd-4981-4fcb-94eb-0b263866bb06 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:55.413864 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8088bd-4981-4fcb-94eb-0b263866bb06 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:55.419142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725f3d9-c93d-424b-84a9-a65f3bc69b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:55.420018 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725f3d9-c93d-424b-84a9-a65f3bc69b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:55.420303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725f3d9-c93d-424b-84a9-a65f3bc69b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:55.420581 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725f3d9-c93d-424b-84a9-a65f3bc69b61 admin admin] PATH_INFO: `/v3/users/d46b939637e045a4b694606a601aea97` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:55.421770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1725f3d9-c93d-424b-84a9-a65f3bc69b61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d46b939637e045a4b694606a601aea97)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:55.423837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1725f3d9-c93d-424b-84a9-a65f3bc69b61 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:55.469765 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1725f3d9-c93d-424b-84a9-a65f3bc69b61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd46b939637e045a4b694606a601aea97'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:55.470513 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1725f3d9-c93d-424b-84a9-a65f3bc69b61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd46b939637e045a4b694606a601aea97'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:55.471113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1725f3d9-c93d-424b-84a9-a65f3bc69b61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd46b939637e045a4b694606a601aea97'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:55.483370 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1725f3d9-c93d-424b-84a9-a65f3bc69b61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd46b939637e045a4b694606a601aea97'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:55.490095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1725f3d9-c93d-424b-84a9-a65f3bc69b61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd46b939637e045a4b694606a601aea97'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:55.493759 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:55.504326 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1102/2147] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:43:55 2022] DELETE /identity/v3/users/d46b939637e045a4b694606a601aea97 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:55.508905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b202739-837a-409e-a4b7-003cd40748bf None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:55.509385 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b202739-837a-409e-a4b7-003cd40748bf None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:55.509505 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b202739-837a-409e-a4b7-003cd40748bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:55.526524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4b202739-837a-409e-a4b7-003cd40748bf None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:55.527710 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4b202739-837a-409e-a4b7-003cd40748bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:55.600192 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1046/2148] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:55.610046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725f3d9-c93d-424b-84a9-a65f3bc69b61 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:55.613360 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1725f3d9-c93d-424b-84a9-a65f3bc69b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:55.696757 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725f3d9-c93d-424b-84a9-a65f3bc69b61 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:55.702407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7bd82-fa92-4d34-a3d3-59efd870c8c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:55.703100 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7bd82-fa92-4d34-a3d3-59efd870c8c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:55.713653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7bd82-fa92-4d34-a3d3-59efd870c8c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:55.713947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7bd82-fa92-4d34-a3d3-59efd870c8c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:55.737540 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7bd82-fa92-4d34-a3d3-59efd870c8c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:55.739895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7bd82-fa92-4d34-a3d3-59efd870c8c8 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:55.752361 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1103/2149] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:43:56.201606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b202739-837a-409e-a4b7-003cd40748bf None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:56.203308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4b202739-837a-409e-a4b7-003cd40748bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:56.236188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b202739-837a-409e-a4b7-003cd40748bf None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:56.251840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7fd5b-77d2-455d-b72c-7fc3977d7eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:56.251840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7fd5b-77d2-455d-b72c-7fc3977d7eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:56.253429 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7fd5b-77d2-455d-b72c-7fc3977d7eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:56.253429 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7fd5b-77d2-455d-b72c-7fc3977d7eaf admin admin] PATH_INFO: `/v3/projects/56e816c04dd44f02bb7a56a3259ba82b` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:56.255746 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d7fd5b-77d2-455d-b72c-7fc3977d7eaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=56e816c04dd44f02bb7a56a3259ba82b)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:56.272653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d7fd5b-77d2-455d-b72c-7fc3977d7eaf admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:56.277595 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be86e8-f1b1-4ca8-be20-dd90a474672f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:56.278229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be86e8-f1b1-4ca8-be20-dd90a474672f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:56.278540 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be86e8-f1b1-4ca8-be20-dd90a474672f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:56.301381 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-48be86e8-f1b1-4ca8-be20-dd90a474672f None None] MFA Rules not processed for user `287144c6346a43d09cdbae463ca9a961`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:56.302741 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-48be86e8-f1b1-4ca8-be20-dd90a474672f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:56.357771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:56.372613 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-98d7fd5b-77d2-455d-b72c-7fc3977d7eaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56e816c04dd44f02bb7a56a3259ba82b', 'request_id': 'req-98d7fd5b-77d2-455d-b72c-7fc3977d7eaf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:56.388258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-98d7fd5b-77d2-455d-b72c-7fc3977d7eaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56e816c04dd44f02bb7a56a3259ba82b', 'request_id': 'req-98d7fd5b-77d2-455d-b72c-7fc3977d7eaf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:56.404639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-98d7fd5b-77d2-455d-b72c-7fc3977d7eaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56e816c04dd44f02bb7a56a3259ba82b', 'request_id': 'req-98d7fd5b-77d2-455d-b72c-7fc3977d7eaf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:56.422614 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1104/2150] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:56.428207 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-98d7fd5b-77d2-455d-b72c-7fc3977d7eaf admin admin] The token cache is being invalidate because project 56e816c04dd44f02bb7a56a3259ba82b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:43:56.428506 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-98d7fd5b-77d2-455d-b72c-7fc3977d7eaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:56.429230 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-98d7fd5b-77d2-455d-b72c-7fc3977d7eaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:43:56.431678 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1047/2151] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:43:56 2022] DELETE /identity/v3/projects/56e816c04dd44f02bb7a56a3259ba82b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:56.436880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944e1e9-21a5-4aa8-a1d2-ac6f3981741d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:56.437248 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944e1e9-21a5-4aa8-a1d2-ac6f3981741d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:56.437665 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944e1e9-21a5-4aa8-a1d2-ac6f3981741d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:56.461986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a944e1e9-21a5-4aa8-a1d2-ac6f3981741d None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:56.463886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a944e1e9-21a5-4aa8-a1d2-ac6f3981741d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:56.550657 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1105/2152] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:56.558686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c34a2-7b01-4dc1-b3af-44ec31c05e0c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:56.559855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c34a2-7b01-4dc1-b3af-44ec31c05e0c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:56.560284 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c34a2-7b01-4dc1-b3af-44ec31c05e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:56.579750 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1d6c34a2-7b01-4dc1-b3af-44ec31c05e0c None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:56.581466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1d6c34a2-7b01-4dc1-b3af-44ec31c05e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:56.595471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1048/2153] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:56.596174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944e1e9-21a5-4aa8-a1d2-ac6f3981741d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:56.608306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944e1e9-21a5-4aa8-a1d2-ac6f3981741d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:56.613204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cad44-9551-492b-a829-a69933365bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:56.614573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cad44-9551-492b-a829-a69933365bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:56.614854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cad44-9551-492b-a829-a69933365bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:56.615236 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cad44-9551-492b-a829-a69933365bf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:56.615903 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801cad44-9551-492b-a829-a69933365bf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:43:56.616240 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801cad44-9551-492b-a829-a69933365bf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:56.617880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801cad44-9551-492b-a829-a69933365bf1 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:56.633925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1106/2154] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:43:56.637139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c34a2-7b01-4dc1-b3af-44ec31c05e0c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:56.647085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c34a2-7b01-4dc1-b3af-44ec31c05e0c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:56.652315 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1b099-71f6-41cb-8b43-e810fd6ec170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:56.652975 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1b099-71f6-41cb-8b43-e810fd6ec170 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:56.653245 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1b099-71f6-41cb-8b43-e810fd6ec170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:56.653515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1b099-71f6-41cb-8b43-e810fd6ec170 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:56.654241 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1b099-71f6-41cb-8b43-e810fd6ec170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:56.656219 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1b099-71f6-41cb-8b43-e810fd6ec170 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:56.687866 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1049/2155] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:43:56 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:56.692110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cad44-9551-492b-a829-a69933365bf1 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:56.703829 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cad44-9551-492b-a829-a69933365bf1 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:56.749796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:43:56.789006 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d99ae-46ea-410c-aad7-4409a3fcd031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:56.789991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d99ae-46ea-410c-aad7-4409a3fcd031 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:56.790549 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d99ae-46ea-410c-aad7-4409a3fcd031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:56.791002 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d99ae-46ea-410c-aad7-4409a3fcd031 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:56.791947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d99ae-46ea-410c-aad7-4409a3fcd031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:56.794466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d99ae-46ea-410c-aad7-4409a3fcd031 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:56.827841 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1107/2156] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:43:56 2022] POST /identity/v3/users => generated 285 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:43:56.832048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1b099-71f6-41cb-8b43-e810fd6ec170 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:56.845562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1b099-71f6-41cb-8b43-e810fd6ec170 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:56.848740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f804c15-9879-4762-bf1d-5ef1d62deb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:56.849370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f804c15-9879-4762-bf1d-5ef1d62deb13 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:56.849658 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f804c15-9879-4762-bf1d-5ef1d62deb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:56.855313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f804c15-9879-4762-bf1d-5ef1d62deb13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:56.855313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f804c15-9879-4762-bf1d-5ef1d62deb13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:56.861819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f804c15-9879-4762-bf1d-5ef1d62deb13 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:56.878369 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1050/2157] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:56.883259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d99ae-46ea-410c-aad7-4409a3fcd031 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:56.902263 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d99ae-46ea-410c-aad7-4409a3fcd031 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:56.907278 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4dbd59-e684-4a48-ae81-0d2a9162df45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:56.907953 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4dbd59-e684-4a48-ae81-0d2a9162df45 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:56.908239 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4dbd59-e684-4a48-ae81-0d2a9162df45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:56.908587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4dbd59-e684-4a48-ae81-0d2a9162df45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:56.909241 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4dbd59-e684-4a48-ae81-0d2a9162df45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:56.912239 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4dbd59-e684-4a48-ae81-0d2a9162df45 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:56.922629 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1108/2158] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:43:56.926724 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f804c15-9879-4762-bf1d-5ef1d62deb13 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:56.937100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f804c15-9879-4762-bf1d-5ef1d62deb13 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:56.942914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e537f0-ef0e-432a-87ac-b695ced50758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:56.943832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e537f0-ef0e-432a-87ac-b695ced50758 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:56.944117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e537f0-ef0e-432a-87ac-b695ced50758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:56.944413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e537f0-ef0e-432a-87ac-b695ced50758 admin admin] PATH_INFO: `/v3/projects/95ef2ec7f9bb433dba1c1ed5bbab7241/users/fd47dce7b01842ed8019635949c06e83/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:56.960552 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e537f0-ef0e-432a-87ac-b695ced50758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ef2ec7f9bb433dba1c1ed5bbab7241, user_id=fd47dce7b01842ed8019635949c06e83, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:56.963306 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e537f0-ef0e-432a-87ac-b695ced50758 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:56.981894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1051/2159] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:43:56 2022] PUT /identity/v3/projects/95ef2ec7f9bb433dba1c1ed5bbab7241/users/fd47dce7b01842ed8019635949c06e83/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:43:56.994412 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb950f7-a2f9-45a7-96c0-d9d41535c38c None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:56.994700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb950f7-a2f9-45a7-96c0-d9d41535c38c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:56.995026 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb950f7-a2f9-45a7-96c0-d9d41535c38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:57.026351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-fdb950f7-a2f9-45a7-96c0-d9d41535c38c None None] MFA Rules not processed for user `fd47dce7b01842ed8019635949c06e83`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:43:57.027807 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fdb950f7-a2f9-45a7-96c0-d9d41535c38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:57.135463 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1109/2160] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:43:56 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:43:57.150553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e537f0-ef0e-432a-87ac-b695ced50758 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:43:57.152342 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-12e537f0-ef0e-432a-87ac-b695ced50758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:43:57.219566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e537f0-ef0e-432a-87ac-b695ced50758 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:43:57.224229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69885ba7-aa89-4970-b514-a4f47037aa8f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebb4da7b00c64fafbb3724bae25b2853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:43:57.224952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69885ba7-aa89-4970-b514-a4f47037aa8f service glance] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:43:57.225283 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69885ba7-aa89-4970-b514-a4f47037aa8f service glance] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:43:57.225609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69885ba7-aa89-4970-b514-a4f47037aa8f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:43:57.234278 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69885ba7-aa89-4970-b514-a4f47037aa8f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:43:57.235699 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69885ba7-aa89-4970-b514-a4f47037aa8f service glance] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:43:57.244733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1052/2161] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:43:57 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:44:02.543350 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358a568-fe81-48e9-be9f-1d7925623233 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:44:02.543894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358a568-fe81-48e9-be9f-1d7925623233 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:44:02.544008 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358a568-fe81-48e9-be9f-1d7925623233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:44:02.562805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a358a568-fe81-48e9-be9f-1d7925623233 None None] MFA Rules not processed for user `bb11db914a57439bada24b5f755a143a`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:44:02.564049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a358a568-fe81-48e9-be9f-1d7925623233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:44:02.623937 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1110/2162] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:44:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:44:06.010111 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69885ba7-aa89-4970-b514-a4f47037aa8f service glance] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:44:06.011523 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-69885ba7-aa89-4970-b514-a4f47037aa8f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:44:06.078356 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69885ba7-aa89-4970-b514-a4f47037aa8f service glance] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:44:06.083330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996876b3-4ad1-426e-847c-672829c8f3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:44:06.084183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996876b3-4ad1-426e-847c-672829c8f3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:44:06.084465 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996876b3-4ad1-426e-847c-672829c8f3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:44:06.084759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996876b3-4ad1-426e-847c-672829c8f3ac admin admin] PATH_INFO: `/v3/users/b650f86088e74077b37a5ae178f90ba0` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:44:06.085937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996876b3-4ad1-426e-847c-672829c8f3ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=b650f86088e74077b37a5ae178f90ba0)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:44:06.087868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996876b3-4ad1-426e-847c-672829c8f3ac admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:44:06.097590 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:44:06.141771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-996876b3-4ad1-426e-847c-672829c8f3ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b650f86088e74077b37a5ae178f90ba0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:06.154561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-996876b3-4ad1-426e-847c-672829c8f3ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b650f86088e74077b37a5ae178f90ba0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:06.161923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-996876b3-4ad1-426e-847c-672829c8f3ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b650f86088e74077b37a5ae178f90ba0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:06.162639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-996876b3-4ad1-426e-847c-672829c8f3ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b650f86088e74077b37a5ae178f90ba0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:06.169703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-996876b3-4ad1-426e-847c-672829c8f3ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b650f86088e74077b37a5ae178f90ba0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:06.175559 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1053/2163] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:44:06 2022] DELETE /identity/v3/users/b650f86088e74077b37a5ae178f90ba0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:44:06.180109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f619f-54dd-4a23-ac5b-31d745d4cd88 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:44:06.180639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f619f-54dd-4a23-ac5b-31d745d4cd88 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:44:06.180739 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f619f-54dd-4a23-ac5b-31d745d4cd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:44:06.200227 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d23f619f-54dd-4a23-ac5b-31d745d4cd88 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:44:06.201444 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d23f619f-54dd-4a23-ac5b-31d745d4cd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:44:06.242566 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:44:06.263577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1111/2164] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:44:06.288263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996876b3-4ad1-426e-847c-672829c8f3ac admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:44:06.289753 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-996876b3-4ad1-426e-847c-672829c8f3ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:44:06.348421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996876b3-4ad1-426e-847c-672829c8f3ac admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:44:06.352782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8bf45-ca1b-4180-bc61-e0f2d1ad5abd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:44:06.353451 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8bf45-ca1b-4180-bc61-e0f2d1ad5abd service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:44:06.353731 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8bf45-ca1b-4180-bc61-e0f2d1ad5abd service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:44:06.354060 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8bf45-ca1b-4180-bc61-e0f2d1ad5abd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:44:06.366325 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8bf45-ca1b-4180-bc61-e0f2d1ad5abd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:44:06.367696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8bf45-ca1b-4180-bc61-e0f2d1ad5abd service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:44:06.377765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1054/2165] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:44:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:44:06.760888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f619f-54dd-4a23-ac5b-31d745d4cd88 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:44:06.762388 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d23f619f-54dd-4a23-ac5b-31d745d4cd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:44:06.775518 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f619f-54dd-4a23-ac5b-31d745d4cd88 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:44:06.781481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d190f1a-582c-4b55-86a7-b95b3c4502a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:44:06.782406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d190f1a-582c-4b55-86a7-b95b3c4502a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:44:06.782696 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d190f1a-582c-4b55-86a7-b95b3c4502a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:44:06.782979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d190f1a-582c-4b55-86a7-b95b3c4502a2 admin admin] PATH_INFO: `/v3/projects/3b8cc00fdb9343d7816dd5a171dd518b` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:44:06.784226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d190f1a-582c-4b55-86a7-b95b3c4502a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b8cc00fdb9343d7816dd5a171dd518b)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:44:06.786239 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d190f1a-582c-4b55-86a7-b95b3c4502a2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:44:06.865687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3d190f1a-582c-4b55-86a7-b95b3c4502a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b8cc00fdb9343d7816dd5a171dd518b', 'request_id': 'req-3d190f1a-582c-4b55-86a7-b95b3c4502a2'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:06.889383 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3d190f1a-582c-4b55-86a7-b95b3c4502a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b8cc00fdb9343d7816dd5a171dd518b', 'request_id': 'req-3d190f1a-582c-4b55-86a7-b95b3c4502a2'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:06.899523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3d190f1a-582c-4b55-86a7-b95b3c4502a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b8cc00fdb9343d7816dd5a171dd518b', 'request_id': 'req-3d190f1a-582c-4b55-86a7-b95b3c4502a2'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:06.921872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3d190f1a-582c-4b55-86a7-b95b3c4502a2 admin admin] The token cache is being invalidate because project 3b8cc00fdb9343d7816dd5a171dd518b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:44:06.921872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3d190f1a-582c-4b55-86a7-b95b3c4502a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:06.921872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3d190f1a-582c-4b55-86a7-b95b3c4502a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:06.921872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1112/2166] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:44:06 2022] DELETE /identity/v3/projects/3b8cc00fdb9343d7816dd5a171dd518b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:44:13.434643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8307a41-508b-49e0-aaa8-207bc8cb12d4 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:44:13.435188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8307a41-508b-49e0-aaa8-207bc8cb12d4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:44:13.435188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8307a41-508b-49e0-aaa8-207bc8cb12d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:44:13.452048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a8307a41-508b-49e0-aaa8-207bc8cb12d4 None None] MFA Rules not processed for user `4a6b3bd40ad744908a05df7410cace52`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:44:13.453284 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a8307a41-508b-49e0-aaa8-207bc8cb12d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:44:13.509941 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1055/2167] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:44:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:44:16.912167 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:44:23.503293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:DELETE=10,keystone:SELECT=73,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:44:27.493720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d190f1a-582c-4b55-86a7-b95b3c4502a2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:44:27.495395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3d190f1a-582c-4b55-86a7-b95b3c4502a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:44:27.568819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d190f1a-582c-4b55-86a7-b95b3c4502a2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:44:27.573536 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41110056-5f97-475f-94d1-6d9befcdcc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:44:27.574464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41110056-5f97-475f-94d1-6d9befcdcc37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:44:27.574753 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41110056-5f97-475f-94d1-6d9befcdcc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:44:27.575103 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41110056-5f97-475f-94d1-6d9befcdcc37 admin admin] PATH_INFO: `/v3/users/fd47dce7b01842ed8019635949c06e83` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:44:27.576273 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41110056-5f97-475f-94d1-6d9befcdcc37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd47dce7b01842ed8019635949c06e83)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:44:27.578286 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41110056-5f97-475f-94d1-6d9befcdcc37 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:44:27.633844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-41110056-5f97-475f-94d1-6d9befcdcc37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd47dce7b01842ed8019635949c06e83'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:27.634520 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-41110056-5f97-475f-94d1-6d9befcdcc37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd47dce7b01842ed8019635949c06e83'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:27.635134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-41110056-5f97-475f-94d1-6d9befcdcc37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd47dce7b01842ed8019635949c06e83'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:27.648247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-41110056-5f97-475f-94d1-6d9befcdcc37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd47dce7b01842ed8019635949c06e83'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:27.655159 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-41110056-5f97-475f-94d1-6d9befcdcc37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd47dce7b01842ed8019635949c06e83'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:27.669481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1113/2168] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:44:27 2022] DELETE /identity/v3/users/fd47dce7b01842ed8019635949c06e83 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:44:27.674349 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804e49d-e1f2-4cb3-a2fe-26f37e053a3c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:44:27.674859 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804e49d-e1f2-4cb3-a2fe-26f37e053a3c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:44:27.674859 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804e49d-e1f2-4cb3-a2fe-26f37e053a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:44:27.693256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-c804e49d-e1f2-4cb3-a2fe-26f37e053a3c None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:44:27.694471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c804e49d-e1f2-4cb3-a2fe-26f37e053a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:44:27.754764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1056/2169] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:44:27.765093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41110056-5f97-475f-94d1-6d9befcdcc37 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:44:27.766621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-41110056-5f97-475f-94d1-6d9befcdcc37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:44:27.824899 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41110056-5f97-475f-94d1-6d9befcdcc37 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:44:27.828685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374a65b-7aed-4131-a8e3-4beaa3cc713e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:44:27.829334 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374a65b-7aed-4131-a8e3-4beaa3cc713e service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:44:27.829619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374a65b-7aed-4131-a8e3-4beaa3cc713e service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:44:27.829895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374a65b-7aed-4131-a8e3-4beaa3cc713e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:44:27.839320 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374a65b-7aed-4131-a8e3-4beaa3cc713e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:44:27.845785 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374a65b-7aed-4131-a8e3-4beaa3cc713e service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:44:27.851400 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1114/2170] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:44:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:44:28.129996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c804e49d-e1f2-4cb3-a2fe-26f37e053a3c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:44:28.131471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c804e49d-e1f2-4cb3-a2fe-26f37e053a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:44:28.143755 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c804e49d-e1f2-4cb3-a2fe-26f37e053a3c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:44:28.148624 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0cc2de-3007-45a4-bf50-45ff75fd79fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:44:28.149633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cc2de-3007-45a4-bf50-45ff75fd79fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:44:28.149979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cc2de-3007-45a4-bf50-45ff75fd79fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:44:28.150346 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cc2de-3007-45a4-bf50-45ff75fd79fc admin admin] PATH_INFO: `/v3/projects/95ef2ec7f9bb433dba1c1ed5bbab7241` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:44:28.151603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0cc2de-3007-45a4-bf50-45ff75fd79fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=95ef2ec7f9bb433dba1c1ed5bbab7241)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:44:28.153750 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0cc2de-3007-45a4-bf50-45ff75fd79fc admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:44:28.211114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2d0cc2de-3007-45a4-bf50-45ff75fd79fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95ef2ec7f9bb433dba1c1ed5bbab7241', 'request_id': 'req-2d0cc2de-3007-45a4-bf50-45ff75fd79fc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:28.216666 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2d0cc2de-3007-45a4-bf50-45ff75fd79fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95ef2ec7f9bb433dba1c1ed5bbab7241', 'request_id': 'req-2d0cc2de-3007-45a4-bf50-45ff75fd79fc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:28.224771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2d0cc2de-3007-45a4-bf50-45ff75fd79fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95ef2ec7f9bb433dba1c1ed5bbab7241', 'request_id': 'req-2d0cc2de-3007-45a4-bf50-45ff75fd79fc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:28.235688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2d0cc2de-3007-45a4-bf50-45ff75fd79fc admin admin] The token cache is being invalidate because project 95ef2ec7f9bb433dba1c1ed5bbab7241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:44:28.235972 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2d0cc2de-3007-45a4-bf50-45ff75fd79fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:28.236595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2d0cc2de-3007-45a4-bf50-45ff75fd79fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:44:28.238159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1057/2171] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:44:28 2022] DELETE /identity/v3/projects/95ef2ec7f9bb433dba1c1ed5bbab7241 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:44:33.916298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374a65b-7aed-4131-a8e3-4beaa3cc713e service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:44:33.924788 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374a65b-7aed-4131-a8e3-4beaa3cc713e service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:44:33.944427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:44:33.983437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b22a2f-63d0-4b6a-9050-27c31c34c888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:44:33.984171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b22a2f-63d0-4b6a-9050-27c31c34c888 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:44:33.984465 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b22a2f-63d0-4b6a-9050-27c31c34c888 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:44:33.984745 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b22a2f-63d0-4b6a-9050-27c31c34c888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:44:33.987404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-09b22a2f-63d0-4b6a-9050-27c31c34c888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:44:34.060683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b22a2f-63d0-4b6a-9050-27c31c34c888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:44:34.062081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b22a2f-63d0-4b6a-9050-27c31c34c888 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:44:34.072224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1115/2172] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:44:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:44:37.888214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb413e57-97d3-42c3-a0ea-918c9fa7d2a4 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:44:37.888763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb413e57-97d3-42c3-a0ea-918c9fa7d2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:44:37.888763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb413e57-97d3-42c3-a0ea-918c9fa7d2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:44:37.906764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-bb413e57-97d3-42c3-a0ea-918c9fa7d2a4 None None] MFA Rules not processed for user `a892b2511463404c818c63e870a23389`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:44:37.908024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bb413e57-97d3-42c3-a0ea-918c9fa7d2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:44:37.963245 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1058/2173] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:44:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:44:41.846193 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419ec0d-e0d7-4d70-85ee-5fab54481dc0 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:44:41.846730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419ec0d-e0d7-4d70-85ee-5fab54481dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:44:41.846837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419ec0d-e0d7-4d70-85ee-5fab54481dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:44:41.869090 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-6419ec0d-e0d7-4d70-85ee-5fab54481dc0 None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:44:41.870627 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6419ec0d-e0d7-4d70-85ee-5fab54481dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:44:41.950082 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1116/2174] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jul 12 13:44:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:44:42.001941 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb413e57-97d3-42c3-a0ea-918c9fa7d2a4 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:44:42.003641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bb413e57-97d3-42c3-a0ea-918c9fa7d2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:44:42.051800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:44:42.105359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb413e57-97d3-42c3-a0ea-918c9fa7d2a4 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:44:42.111137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899e0ec-7438-41de-b9d3-ec4daa8439f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:44:42.111550 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899e0ec-7438-41de-b9d3-ec4daa8439f7 service swift] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:44:42.111672 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899e0ec-7438-41de-b9d3-ec4daa8439f7 service swift] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:44:42.112065 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899e0ec-7438-41de-b9d3-ec4daa8439f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:44:42.122742 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899e0ec-7438-41de-b9d3-ec4daa8439f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:44:42.124514 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899e0ec-7438-41de-b9d3-ec4daa8439f7 service swift] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:44:42.131645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e63694-f3c0-43c3-ab1d-f4d7e468ede9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:44:42.132036 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e63694-f3c0-43c3-ab1d-f4d7e468ede9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:44:42.132283 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e63694-f3c0-43c3-ab1d-f4d7e468ede9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:44:42.142394 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1059/2175] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:44:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:44:42.155067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-09e63694-f3c0-43c3-ab1d-f4d7e468ede9 None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:44:42.156610 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-09e63694-f3c0-43c3-ab1d-f4d7e468ede9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:44:42.168439 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1117/2176] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jul 12 13:44:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:44:42.193266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899e0ec-7438-41de-b9d3-ec4daa8439f7 service swift] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:44:42.215687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899e0ec-7438-41de-b9d3-ec4daa8439f7 service swift] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:44:42.240450 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631778d-208c-4fb7-bdff-94056f183d64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:44:42.241297 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631778d-208c-4fb7-bdff-94056f183d64 service swift] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:44:42.241759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631778d-208c-4fb7-bdff-94056f183d64 service swift] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:44:42.242249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631778d-208c-4fb7-bdff-94056f183d64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:44:42.264993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631778d-208c-4fb7-bdff-94056f183d64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:44:42.266653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631778d-208c-4fb7-bdff-94056f183d64 service swift] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:44:42.298471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1060/2177] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:44:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 13:44:52.150389 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=68 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:44:52.282759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=25 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:44:59.263598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e63694-f3c0-43c3-ab1d-f4d7e468ede9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:44:59.265677 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-09e63694-f3c0-43c3-ab1d-f4d7e468ede9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:44:59.365935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e63694-f3c0-43c3-ab1d-f4d7e468ede9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:44:59.371361 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90afecd4-96a7-4dc5-ba7a-a46ce96d25fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1308deea1ea4629a1ac22d4b903ad4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:44:59.372231 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afecd4-96a7-4dc5-ba7a-a46ce96d25fe service placement] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:44:59.372499 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afecd4-96a7-4dc5-ba7a-a46ce96d25fe service placement] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:44:59.372886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afecd4-96a7-4dc5-ba7a-a46ce96d25fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:44:59.375398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-90afecd4-96a7-4dc5-ba7a-a46ce96d25fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:44:59.393705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90afecd4-96a7-4dc5-ba7a-a46ce96d25fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:44:59.396453 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90afecd4-96a7-4dc5-ba7a-a46ce96d25fe service placement] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:44:59.409640 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1118/2178] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:44:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:45:01.317436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12fe70-6481-4f7d-8797-e57861f14ece None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:01.318445 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12fe70-6481-4f7d-8797-e57861f14ece None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:01.318895 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12fe70-6481-4f7d-8797-e57861f14ece None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:01.320769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1061/2179] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jul 12 13:45:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:45:01.328223 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda21411-24dd-4223-a8d2-dca02019040d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:01.328767 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda21411-24dd-4223-a8d2-dca02019040d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:01.328767 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda21411-24dd-4223-a8d2-dca02019040d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:01.361779 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-fda21411-24dd-4223-a8d2-dca02019040d None None] MFA Rules not processed for user `0d3915cde0d04bcfb9b5ce8c89af1e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:45:01.363692 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fda21411-24dd-4223-a8d2-dca02019040d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:01.462587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1119/2180] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jul 12 13:45:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:45:01.470480 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12fe70-6481-4f7d-8797-e57861f14ece None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:01.473089 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-aa12fe70-6481-4f7d-8797-e57861f14ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:01.539165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12fe70-6481-4f7d-8797-e57861f14ece None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:01.542705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d64e7-cfd3-4ebf-be55-3d133b3d978d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebb4da7b00c64fafbb3724bae25b2853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:01.543359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d64e7-cfd3-4ebf-be55-3d133b3d978d service glance] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:01.543636 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d64e7-cfd3-4ebf-be55-3d133b3d978d service glance] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:01.543907 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d64e7-cfd3-4ebf-be55-3d133b3d978d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:01.554565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d64e7-cfd3-4ebf-be55-3d133b3d978d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:01.556008 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d64e7-cfd3-4ebf-be55-3d133b3d978d service glance] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:01.572968 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1062/2181] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:45:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:45:01.655584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb914a-b42e-4a95-a963-39eddf7b327f None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:01.656178 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb914a-b42e-4a95-a963-39eddf7b327f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:01.656486 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb914a-b42e-4a95-a963-39eddf7b327f None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:01.658406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1120/2182] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jul 12 13:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:45:01.663304 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac41b5-282b-4edb-af4b-215f520bd68a None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:01.663696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac41b5-282b-4edb-af4b-215f520bd68a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:01.663918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac41b5-282b-4edb-af4b-215f520bd68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:01.683361 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0eac41b5-282b-4edb-af4b-215f520bd68a None None] MFA Rules not processed for user `3e833722fe4e493e9ddc4b275632536f`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:45:01.684792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0eac41b5-282b-4edb-af4b-215f520bd68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:01.691872 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1063/2183] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jul 12 13:45:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:45:01.706699 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb914a-b42e-4a95-a963-39eddf7b327f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:01.709675 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-facb914a-b42e-4a95-a963-39eddf7b327f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:01.725936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb914a-b42e-4a95-a963-39eddf7b327f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:01.729708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5804b6-6933-425a-8594-5b35d576235f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:01.730393 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5804b6-6933-425a-8594-5b35d576235f service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:01.730682 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5804b6-6933-425a-8594-5b35d576235f service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:01.730962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5804b6-6933-425a-8594-5b35d576235f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:01.738339 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5804b6-6933-425a-8594-5b35d576235f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:01.739785 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5804b6-6933-425a-8594-5b35d576235f service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:01.748384 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1121/2184] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:45:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:45:01.835707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0231592-7fa5-4214-bd0e-5eef6322c46e None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:01.835995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0231592-7fa5-4214-bd0e-5eef6322c46e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:01.836292 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0231592-7fa5-4214-bd0e-5eef6322c46e None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:01.837654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1064/2185] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jul 12 13:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:45:01.842647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0bbb9-4fe1-40cb-968e-bbf5c163afe8 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:01.842962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0bbb9-4fe1-40cb-968e-bbf5c163afe8 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:01.843241 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0bbb9-4fe1-40cb-968e-bbf5c163afe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:01.861890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d8c0bbb9-4fe1-40cb-968e-bbf5c163afe8 None None] MFA Rules not processed for user `0d3915cde0d04bcfb9b5ce8c89af1e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:45:01.863141 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d8c0bbb9-4fe1-40cb-968e-bbf5c163afe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:01.873096 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1122/2186] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jul 12 13:45:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:45:01.884164 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0231592-7fa5-4214-bd0e-5eef6322c46e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:01.895996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0231592-7fa5-4214-bd0e-5eef6322c46e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:01.899589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76af1b5-954e-4917-88c7-295c57a704ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:01.900215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76af1b5-954e-4917-88c7-295c57a704ee service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:01.900631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76af1b5-954e-4917-88c7-295c57a704ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:01.900789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76af1b5-954e-4917-88c7-295c57a704ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:01.910857 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76af1b5-954e-4917-88c7-295c57a704ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:01.912310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76af1b5-954e-4917-88c7-295c57a704ee service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:01.922943 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1065/2187] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:45:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:45:03.136884 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d38be0-1139-4d48-b83b-8a40c5c70128 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:03.137350 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d38be0-1139-4d48-b83b-8a40c5c70128 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:03.137442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d38be0-1139-4d48-b83b-8a40c5c70128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:03.155304 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-26d38be0-1139-4d48-b83b-8a40c5c70128 None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:45:03.156507 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-26d38be0-1139-4d48-b83b-8a40c5c70128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:03.164392 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1123/2188] 10.222.0.52 () {58 vars in 967 bytes} [Tue Jul 12 13:45:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:45:03.176574 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76af1b5-954e-4917-88c7-295c57a704ee service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:03.187818 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76af1b5-954e-4917-88c7-295c57a704ee service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:03.193073 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d35f1-791e-4dd6-b280-330421b1e30c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:03.193705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d35f1-791e-4dd6-b280-330421b1e30c service swift] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:03.194017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d35f1-791e-4dd6-b280-330421b1e30c service swift] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:03.194299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d35f1-791e-4dd6-b280-330421b1e30c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:03.202345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d35f1-791e-4dd6-b280-330421b1e30c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:03.203923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d35f1-791e-4dd6-b280-330421b1e30c service swift] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:03.212566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1066/2189] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:45:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:45:04.289421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b81e1-6c20-421d-96a1-ec83d37332a9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:04.290292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b81e1-6c20-421d-96a1-ec83d37332a9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:04.290576 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b81e1-6c20-421d-96a1-ec83d37332a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:04.309051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-af9b81e1-6c20-421d-96a1-ec83d37332a9 None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:45:04.310292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-af9b81e1-6c20-421d-96a1-ec83d37332a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:04.317501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1124/2190] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jul 12 13:45:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:45:04.329545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d35f1-791e-4dd6-b280-330421b1e30c service swift] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:04.341840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d35f1-791e-4dd6-b280-330421b1e30c service swift] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:04.347913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81237284-57fc-44ea-880d-086e014e8947 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:04.348557 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81237284-57fc-44ea-880d-086e014e8947 service swift] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:04.348844 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81237284-57fc-44ea-880d-086e014e8947 service swift] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:04.349153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81237284-57fc-44ea-880d-086e014e8947 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:04.357821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81237284-57fc-44ea-880d-086e014e8947 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:04.359451 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81237284-57fc-44ea-880d-086e014e8947 service swift] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:04.368309 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1067/2191] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:45:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:45:05.286645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b81e1-6c20-421d-96a1-ec83d37332a9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:05.288531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-af9b81e1-6c20-421d-96a1-ec83d37332a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:05.364805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:45:05.398019 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b81e1-6c20-421d-96a1-ec83d37332a9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:05.403097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6f062-73eb-4883-ad8b-a5a1590a716e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:05.403981 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6f062-73eb-4883-ad8b-a5a1590a716e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:05.404268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6f062-73eb-4883-ad8b-a5a1590a716e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:05.404538 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6f062-73eb-4883-ad8b-a5a1590a716e admin admin] PATH_INFO: `/v3/users/287144c6346a43d09cdbae463ca9a961` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:05.405785 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6f062-73eb-4883-ad8b-a5a1590a716e admin admin] RBAC: Authorizing `identity:delete_user(user_id=287144c6346a43d09cdbae463ca9a961)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:05.407756 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6f062-73eb-4883-ad8b-a5a1590a716e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:05.461412 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7fb6f062-73eb-4883-ad8b-a5a1590a716e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '287144c6346a43d09cdbae463ca9a961'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:05.462184 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7fb6f062-73eb-4883-ad8b-a5a1590a716e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '287144c6346a43d09cdbae463ca9a961'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:05.462870 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7fb6f062-73eb-4883-ad8b-a5a1590a716e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '287144c6346a43d09cdbae463ca9a961'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:05.475282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7fb6f062-73eb-4883-ad8b-a5a1590a716e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '287144c6346a43d09cdbae463ca9a961'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:05.482310 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7fb6f062-73eb-4883-ad8b-a5a1590a716e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '287144c6346a43d09cdbae463ca9a961'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:05.497813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1125/2192] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:45:05 2022] DELETE /identity/v3/users/287144c6346a43d09cdbae463ca9a961 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:45:05.683274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81237284-57fc-44ea-880d-086e014e8947 service swift] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:05.715861 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81237284-57fc-44ea-880d-086e014e8947 service swift] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:05.792894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4cdff-be9c-4675-bbb2-4f79c50d2795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:05.793808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4cdff-be9c-4675-bbb2-4f79c50d2795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:05.794121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4cdff-be9c-4675-bbb2-4f79c50d2795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:05.794395 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4cdff-be9c-4675-bbb2-4f79c50d2795 admin admin] PATH_INFO: `/v3/projects/4f3ab1e8d5fd4ac7b34c60a4a3cf9c45` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:05.795597 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4cdff-be9c-4675-bbb2-4f79c50d2795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f3ab1e8d5fd4ac7b34c60a4a3cf9c45)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:05.797562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4cdff-be9c-4675-bbb2-4f79c50d2795 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:05.857751 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-75f4cdff-be9c-4675-bbb2-4f79c50d2795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f3ab1e8d5fd4ac7b34c60a4a3cf9c45', 'request_id': 'req-75f4cdff-be9c-4675-bbb2-4f79c50d2795'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:05.864363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-75f4cdff-be9c-4675-bbb2-4f79c50d2795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f3ab1e8d5fd4ac7b34c60a4a3cf9c45', 'request_id': 'req-75f4cdff-be9c-4675-bbb2-4f79c50d2795'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:05.866062 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:45:05.873495 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-75f4cdff-be9c-4675-bbb2-4f79c50d2795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f3ab1e8d5fd4ac7b34c60a4a3cf9c45', 'request_id': 'req-75f4cdff-be9c-4675-bbb2-4f79c50d2795'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:05.885520 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-75f4cdff-be9c-4675-bbb2-4f79c50d2795 admin admin] The token cache is being invalidate because project 4f3ab1e8d5fd4ac7b34c60a4a3cf9c45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:45:05.885747 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-75f4cdff-be9c-4675-bbb2-4f79c50d2795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:05.886520 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-75f4cdff-be9c-4675-bbb2-4f79c50d2795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:05.888375 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1068/2193] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:45:05 2022] DELETE /identity/v3/projects/4f3ab1e8d5fd4ac7b34c60a4a3cf9c45 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:45:05.893302 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0d4e3-2782-46ee-b6a5-2000b6f39325 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:05.893601 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0d4e3-2782-46ee-b6a5-2000b6f39325 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:05.893883 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0d4e3-2782-46ee-b6a5-2000b6f39325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:05.913252 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4af0d4e3-2782-46ee-b6a5-2000b6f39325 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:45:05.914516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4af0d4e3-2782-46ee-b6a5-2000b6f39325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:05.990198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1126/2194] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:45:05.997697 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898db02-afe6-4cf9-b4fe-e949f502a0bc None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:05.998014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898db02-afe6-4cf9-b4fe-e949f502a0bc None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:05.998298 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898db02-afe6-4cf9-b4fe-e949f502a0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:06.019061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-5898db02-afe6-4cf9-b4fe-e949f502a0bc None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:45:06.020501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5898db02-afe6-4cf9-b4fe-e949f502a0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:06.043494 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1069/2195] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:45:06.047738 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0d4e3-2782-46ee-b6a5-2000b6f39325 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:06.058608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0d4e3-2782-46ee-b6a5-2000b6f39325 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:06.065332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4573c-0a19-43ee-bb59-3faa6d5729d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:06.066768 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4573c-0a19-43ee-bb59-3faa6d5729d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:06.067057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4573c-0a19-43ee-bb59-3faa6d5729d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:06.067332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4573c-0a19-43ee-bb59-3faa6d5729d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:06.068368 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4573c-0a19-43ee-bb59-3faa6d5729d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:45:06.068680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4573c-0a19-43ee-bb59-3faa6d5729d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:06.070501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4573c-0a19-43ee-bb59-3faa6d5729d6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:06.082889 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1127/2196] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:45:06.087475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898db02-afe6-4cf9-b4fe-e949f502a0bc None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:06.097095 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898db02-afe6-4cf9-b4fe-e949f502a0bc None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:06.102384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016df99f-0787-4fda-8a92-a4bd3d949b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:06.103126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016df99f-0787-4fda-8a92-a4bd3d949b26 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:06.103496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016df99f-0787-4fda-8a92-a4bd3d949b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:06.103841 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016df99f-0787-4fda-8a92-a4bd3d949b26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:06.104617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016df99f-0787-4fda-8a92-a4bd3d949b26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:06.106786 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016df99f-0787-4fda-8a92-a4bd3d949b26 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:06.141104 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1070/2197] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:45:06 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:45:06.145261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4573c-0a19-43ee-bb59-3faa6d5729d6 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:06.155470 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4573c-0a19-43ee-bb59-3faa6d5729d6 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:06.220165 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:45:06.229107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c69ea7-1462-424f-85a9-522845b0f43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:06.230013 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c69ea7-1462-424f-85a9-522845b0f43a admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:06.234417 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c69ea7-1462-424f-85a9-522845b0f43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:06.234417 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c69ea7-1462-424f-85a9-522845b0f43a admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:06.234417 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c69ea7-1462-424f-85a9-522845b0f43a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:06.234417 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c69ea7-1462-424f-85a9-522845b0f43a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:06.273716 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1128/2198] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:45:06 2022] POST /identity/v3/users => generated 285 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:45:06.275144 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016df99f-0787-4fda-8a92-a4bd3d949b26 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:06.286263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016df99f-0787-4fda-8a92-a4bd3d949b26 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:06.291058 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e54ac4-1b63-40df-9f36-380fe7edc0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:06.291830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e54ac4-1b63-40df-9f36-380fe7edc0aa admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:06.292123 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e54ac4-1b63-40df-9f36-380fe7edc0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:06.292436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e54ac4-1b63-40df-9f36-380fe7edc0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:06.293201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e54ac4-1b63-40df-9f36-380fe7edc0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:06.294831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e54ac4-1b63-40df-9f36-380fe7edc0aa admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:06.306542 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1071/2199] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:45:06.310439 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c69ea7-1462-424f-85a9-522845b0f43a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:06.323526 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c69ea7-1462-424f-85a9-522845b0f43a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:06.331743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dde06a-ac1b-4306-ad3c-b556601ec9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:06.333254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dde06a-ac1b-4306-ad3c-b556601ec9d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:06.333601 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dde06a-ac1b-4306-ad3c-b556601ec9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:06.333852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dde06a-ac1b-4306-ad3c-b556601ec9d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:06.334650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dde06a-ac1b-4306-ad3c-b556601ec9d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:06.336908 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dde06a-ac1b-4306-ad3c-b556601ec9d5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:06.364237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1129/2200] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:45:06.365629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e54ac4-1b63-40df-9f36-380fe7edc0aa admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:06.380567 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e54ac4-1b63-40df-9f36-380fe7edc0aa admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:06.385731 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f53565f-d461-4a90-8188-e2ac5896a73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:06.387021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53565f-d461-4a90-8188-e2ac5896a73a admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:06.387420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53565f-d461-4a90-8188-e2ac5896a73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:06.387820 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53565f-d461-4a90-8188-e2ac5896a73a admin admin] PATH_INFO: `/v3/projects/c35faaf809f14de288ccdb191de38737/users/601e811b8b8848c990a9d4789761c0e4/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:06.405347 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f53565f-d461-4a90-8188-e2ac5896a73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c35faaf809f14de288ccdb191de38737, user_id=601e811b8b8848c990a9d4789761c0e4, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:06.408852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f53565f-d461-4a90-8188-e2ac5896a73a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:06.427582 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1072/2201] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:45:06 2022] PUT /identity/v3/projects/c35faaf809f14de288ccdb191de38737/users/601e811b8b8848c990a9d4789761c0e4/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:45:06.437951 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ffc3-7374-4137-8855-2ead3cc6ce8b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:06.438587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ffc3-7374-4137-8855-2ead3cc6ce8b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:06.438587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ffc3-7374-4137-8855-2ead3cc6ce8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:06.485005 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3e44ffc3-7374-4137-8855-2ead3cc6ce8b None None] MFA Rules not processed for user `601e811b8b8848c990a9d4789761c0e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:45:06.486296 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3e44ffc3-7374-4137-8855-2ead3cc6ce8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:06.567857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1130/2202] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:45:06 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:45:06.579105 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f53565f-d461-4a90-8188-e2ac5896a73a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:06.581044 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3f53565f-d461-4a90-8188-e2ac5896a73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:06.662496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f53565f-d461-4a90-8188-e2ac5896a73a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:06.666183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6356d-3fe7-4f0e-bf27-f8c6e54e601a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebb4da7b00c64fafbb3724bae25b2853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:06.666823 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6356d-3fe7-4f0e-bf27-f8c6e54e601a service glance] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:06.667114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6356d-3fe7-4f0e-bf27-f8c6e54e601a service glance] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:06.667392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6356d-3fe7-4f0e-bf27-f8c6e54e601a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:06.676752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6356d-3fe7-4f0e-bf27-f8c6e54e601a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:06.678247 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6356d-3fe7-4f0e-bf27-f8c6e54e601a service glance] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:06.694221 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1073/2203] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:45:06 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:45:16.564932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=41,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:45:16.683257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=57 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:45:32.302466 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e76ab1-7fe7-4c2e-9fe0-4c8ea9495ffe None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:32.302925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e76ab1-7fe7-4c2e-9fe0-4c8ea9495ffe None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:32.303051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e76ab1-7fe7-4c2e-9fe0-4c8ea9495ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:32.321315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a1e76ab1-7fe7-4c2e-9fe0-4c8ea9495ffe None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:45:32.322542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a1e76ab1-7fe7-4c2e-9fe0-4c8ea9495ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:32.376043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1131/2204] 10.222.0.52 () {58 vars in 967 bytes} [Tue Jul 12 13:45:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:45:32.388682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6356d-3fe7-4f0e-bf27-f8c6e54e601a service glance] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:32.390125 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-dbf6356d-3fe7-4f0e-bf27-f8c6e54e601a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:32.459274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6356d-3fe7-4f0e-bf27-f8c6e54e601a service glance] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:32.464839 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972569e0-89a8-45f2-a000-33feebfb4a4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:32.465474 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972569e0-89a8-45f2-a000-33feebfb4a4c service swift] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:32.465760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972569e0-89a8-45f2-a000-33feebfb4a4c service swift] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:32.466075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972569e0-89a8-45f2-a000-33feebfb4a4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:32.473997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972569e0-89a8-45f2-a000-33feebfb4a4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:32.475620 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972569e0-89a8-45f2-a000-33feebfb4a4c service swift] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:32.483626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1074/2205] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:45:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:45:42.369221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=25 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:45:42.480358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=32 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:45:43.091202 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e76ab1-7fe7-4c2e-9fe0-4c8ea9495ffe None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:43.153713 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e76ab1-7fe7-4c2e-9fe0-4c8ea9495ffe None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:43.158226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d92a09-99cf-4b13-b663-26115950b266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:43.159089 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d92a09-99cf-4b13-b663-26115950b266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:43.159369 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d92a09-99cf-4b13-b663-26115950b266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:43.159643 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d92a09-99cf-4b13-b663-26115950b266 admin admin] PATH_INFO: `/v3/users/601e811b8b8848c990a9d4789761c0e4` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:43.160835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d92a09-99cf-4b13-b663-26115950b266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=601e811b8b8848c990a9d4789761c0e4)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:43.162795 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d92a09-99cf-4b13-b663-26115950b266 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:43.206950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e0d92a09-99cf-4b13-b663-26115950b266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '601e811b8b8848c990a9d4789761c0e4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:43.207615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e0d92a09-99cf-4b13-b663-26115950b266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '601e811b8b8848c990a9d4789761c0e4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:43.208242 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e0d92a09-99cf-4b13-b663-26115950b266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '601e811b8b8848c990a9d4789761c0e4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:43.220640 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e0d92a09-99cf-4b13-b663-26115950b266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '601e811b8b8848c990a9d4789761c0e4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:43.227387 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e0d92a09-99cf-4b13-b663-26115950b266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '601e811b8b8848c990a9d4789761c0e4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:43.240578 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1132/2206] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:45:43 2022] DELETE /identity/v3/users/601e811b8b8848c990a9d4789761c0e4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:45:43.245543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e309c-08eb-406c-8274-f80d86099092 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:43.245822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e309c-08eb-406c-8274-f80d86099092 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:43.246146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e309c-08eb-406c-8274-f80d86099092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:43.263798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e38e309c-08eb-406c-8274-f80d86099092 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:45:43.265015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e38e309c-08eb-406c-8274-f80d86099092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:43.322576 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1075/2207] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:45:43.332365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d92a09-99cf-4b13-b663-26115950b266 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:43.333827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e0d92a09-99cf-4b13-b663-26115950b266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:43.390897 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d92a09-99cf-4b13-b663-26115950b266 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:43.394751 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70115775-66ed-4752-bfc1-e8ec8358216d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:43.395418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115775-66ed-4752-bfc1-e8ec8358216d service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:43.395702 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115775-66ed-4752-bfc1-e8ec8358216d service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:43.395976 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115775-66ed-4752-bfc1-e8ec8358216d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:43.405326 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70115775-66ed-4752-bfc1-e8ec8358216d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:43.406793 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70115775-66ed-4752-bfc1-e8ec8358216d service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:43.416335 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1133/2208] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:45:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:45:43.681332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e309c-08eb-406c-8274-f80d86099092 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:43.682784 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e38e309c-08eb-406c-8274-f80d86099092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:43.696100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e309c-08eb-406c-8274-f80d86099092 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:43.701685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9ee03-fde2-4d3a-83b7-e1492306e99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:43.702584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9ee03-fde2-4d3a-83b7-e1492306e99e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:43.702868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9ee03-fde2-4d3a-83b7-e1492306e99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:43.703147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9ee03-fde2-4d3a-83b7-e1492306e99e admin admin] PATH_INFO: `/v3/projects/c35faaf809f14de288ccdb191de38737` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:43.704365 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9ee03-fde2-4d3a-83b7-e1492306e99e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c35faaf809f14de288ccdb191de38737)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:43.706660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9ee03-fde2-4d3a-83b7-e1492306e99e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:43.753695 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-34f9ee03-fde2-4d3a-83b7-e1492306e99e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c35faaf809f14de288ccdb191de38737', 'request_id': 'req-34f9ee03-fde2-4d3a-83b7-e1492306e99e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:43.759329 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-34f9ee03-fde2-4d3a-83b7-e1492306e99e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c35faaf809f14de288ccdb191de38737', 'request_id': 'req-34f9ee03-fde2-4d3a-83b7-e1492306e99e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:43.767671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-34f9ee03-fde2-4d3a-83b7-e1492306e99e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c35faaf809f14de288ccdb191de38737', 'request_id': 'req-34f9ee03-fde2-4d3a-83b7-e1492306e99e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:43.778398 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-34f9ee03-fde2-4d3a-83b7-e1492306e99e admin admin] The token cache is being invalidate because project c35faaf809f14de288ccdb191de38737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:45:43.778687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-34f9ee03-fde2-4d3a-83b7-e1492306e99e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:43.779319 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-34f9ee03-fde2-4d3a-83b7-e1492306e99e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:45:43.780852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1076/2209] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:45:43 2022] DELETE /identity/v3/projects/c35faaf809f14de288ccdb191de38737 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:45:43.785726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8fed6-78d7-4366-8a23-d2b04c51b042 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:43.786050 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8fed6-78d7-4366-8a23-d2b04c51b042 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:43.786372 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8fed6-78d7-4366-8a23-d2b04c51b042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:43.805071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b7a8fed6-78d7-4366-8a23-d2b04c51b042 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:45:43.806321 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b7a8fed6-78d7-4366-8a23-d2b04c51b042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:43.822640 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:45:43.867765 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1134/2210] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:45:43.875584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b95fa-b319-438e-b23b-22c31fd984c4 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:43.875878 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b95fa-b319-438e-b23b-22c31fd984c4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:43.876160 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b95fa-b319-438e-b23b-22c31fd984c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:43.894522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ee2b95fa-b319-438e-b23b-22c31fd984c4 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:45:43.895726 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ee2b95fa-b319-438e-b23b-22c31fd984c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:43.903655 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1077/2211] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:45:43.907550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8fed6-78d7-4366-8a23-d2b04c51b042 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:43.917584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8fed6-78d7-4366-8a23-d2b04c51b042 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:43.922162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3f778-cb6d-4c3e-be5f-1262a190f5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:43.922806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3f778-cb6d-4c3e-be5f-1262a190f5e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:43.923110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3f778-cb6d-4c3e-be5f-1262a190f5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:43.923381 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3f778-cb6d-4c3e-be5f-1262a190f5e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:43.924094 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3f778-cb6d-4c3e-be5f-1262a190f5e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:45:43.924396 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3f778-cb6d-4c3e-be5f-1262a190f5e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:43.926102 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3f778-cb6d-4c3e-be5f-1262a190f5e5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:43.937807 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1135/2212] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:45:43.941905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b95fa-b319-438e-b23b-22c31fd984c4 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:43.952151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b95fa-b319-438e-b23b-22c31fd984c4 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:43.956978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd5b99-9a40-4f3c-8d13-ade21c58acbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:43.957791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd5b99-9a40-4f3c-8d13-ade21c58acbc admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:43.958095 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd5b99-9a40-4f3c-8d13-ade21c58acbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:43.958517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd5b99-9a40-4f3c-8d13-ade21c58acbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:43.959354 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd5b99-9a40-4f3c-8d13-ade21c58acbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:43.961252 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd5b99-9a40-4f3c-8d13-ade21c58acbc admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:43.990973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1078/2213] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:45:43 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:45:43.992725 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3f778-cb6d-4c3e-be5f-1262a190f5e5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:44.003751 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3f778-cb6d-4c3e-be5f-1262a190f5e5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:44.063750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96f7ce-0ddb-42c1-af2a-d884d4695d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:44.064422 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96f7ce-0ddb-42c1-af2a-d884d4695d81 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:44.064706 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96f7ce-0ddb-42c1-af2a-d884d4695d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:44.064985 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96f7ce-0ddb-42c1-af2a-d884d4695d81 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:44.065779 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96f7ce-0ddb-42c1-af2a-d884d4695d81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:44.067741 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96f7ce-0ddb-42c1-af2a-d884d4695d81 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:44.091298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1136/2214] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:45:43 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:45:44.092492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd5b99-9a40-4f3c-8d13-ade21c58acbc admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:44.103578 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd5b99-9a40-4f3c-8d13-ade21c58acbc admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:44.108323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46f7e5-bc7e-48e6-aab1-4f52a0c61a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:44.108982 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46f7e5-bc7e-48e6-aab1-4f52a0c61a8b admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:44.109269 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46f7e5-bc7e-48e6-aab1-4f52a0c61a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:44.109549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46f7e5-bc7e-48e6-aab1-4f52a0c61a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:44.110314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46f7e5-bc7e-48e6-aab1-4f52a0c61a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:44.111919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46f7e5-bc7e-48e6-aab1-4f52a0c61a8b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:44.122321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1079/2215] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:45:44.126236 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96f7ce-0ddb-42c1-af2a-d884d4695d81 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:44.136740 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96f7ce-0ddb-42c1-af2a-d884d4695d81 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:44.141586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1062f-601b-4468-a693-034d6c0146d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:44.142355 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1062f-601b-4468-a693-034d6c0146d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:44.142709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1062f-601b-4468-a693-034d6c0146d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:44.143041 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1062f-601b-4468-a693-034d6c0146d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:44.143835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1062f-601b-4468-a693-034d6c0146d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:44.145499 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1062f-601b-4468-a693-034d6c0146d0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:44.156642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1137/2216] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:45:44.160752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46f7e5-bc7e-48e6-aab1-4f52a0c61a8b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:44.171257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46f7e5-bc7e-48e6-aab1-4f52a0c61a8b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:44.175933 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94cb2b-6860-4526-b1e6-89093185977d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:44.176804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94cb2b-6860-4526-b1e6-89093185977d admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:44.177090 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94cb2b-6860-4526-b1e6-89093185977d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:44.177487 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94cb2b-6860-4526-b1e6-89093185977d admin admin] PATH_INFO: `/v3/projects/adf5aa6b70414b8f81904a32fa4fa1ab/users/1a50af3c7d004e78a99875e1f36a9508/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:44.193124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94cb2b-6860-4526-b1e6-89093185977d admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf5aa6b70414b8f81904a32fa4fa1ab, user_id=1a50af3c7d004e78a99875e1f36a9508, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:44.195845 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94cb2b-6860-4526-b1e6-89093185977d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:44.209791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1080/2217] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:45:44 2022] PUT /identity/v3/projects/adf5aa6b70414b8f81904a32fa4fa1ab/users/1a50af3c7d004e78a99875e1f36a9508/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:45:44.215396 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a24ae-8b0c-4d97-a641-78ca61718d1d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:44.215684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a24ae-8b0c-4d97-a641-78ca61718d1d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:44.216000 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a24ae-8b0c-4d97-a641-78ca61718d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:44.237700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-715a24ae-8b0c-4d97-a641-78ca61718d1d None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:45:44.238980 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-715a24ae-8b0c-4d97-a641-78ca61718d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:44.298921 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1138/2218] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:45:44.308314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94cb2b-6860-4526-b1e6-89093185977d admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:44.310181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ff94cb2b-6860-4526-b1e6-89093185977d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:44.332481 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:45:44.370956 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94cb2b-6860-4526-b1e6-89093185977d admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:44.374883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecda023-ed83-486d-8054-56aba25eed94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:44.375467 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecda023-ed83-486d-8054-56aba25eed94 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:44.375753 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecda023-ed83-486d-8054-56aba25eed94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:44.376032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecda023-ed83-486d-8054-56aba25eed94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:44.385708 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecda023-ed83-486d-8054-56aba25eed94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:44.387181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecda023-ed83-486d-8054-56aba25eed94 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:44.396507 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1081/2219] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:45:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:45:44.966237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab8b41-666e-44bd-9a6d-e71655ce85b9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:44.966738 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab8b41-666e-44bd-9a6d-e71655ce85b9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:44.966861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab8b41-666e-44bd-9a6d-e71655ce85b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:44.993657 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9fab8b41-666e-44bd-9a6d-e71655ce85b9 None None] MFA Rules not processed for user `1a50af3c7d004e78a99875e1f36a9508`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:45:44.994931 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9fab8b41-666e-44bd-9a6d-e71655ce85b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:45.013809 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:45:45.075383 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1139/2220] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:45:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:45:45.086188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecda023-ed83-486d-8054-56aba25eed94 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:45:45.087583 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fecda023-ed83-486d-8054-56aba25eed94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:45.147422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecda023-ed83-486d-8054-56aba25eed94 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:45:45.151126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e94ad-c183-47cb-8fc0-27ba4e5cce88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:45:45.151720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e94ad-c183-47cb-8fc0-27ba4e5cce88 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:45.152006 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e94ad-c183-47cb-8fc0-27ba4e5cce88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:45.152285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e94ad-c183-47cb-8fc0-27ba4e5cce88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:45.160640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e94ad-c183-47cb-8fc0-27ba4e5cce88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:45:45.162108 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e94ad-c183-47cb-8fc0-27ba4e5cce88 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:45:45.171278 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1082/2221] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:45:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:45:46.531898 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a9d61-fe02-4868-b589-702008f84d71 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:45:46.532437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a9d61-fe02-4868-b589-702008f84d71 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:45:46.532437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a9d61-fe02-4868-b589-702008f84d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:45:46.549527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1d5a9d61-fe02-4868-b589-702008f84d71 None None] MFA Rules not processed for user `1a50af3c7d004e78a99875e1f36a9508`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:45:46.550765 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1d5a9d61-fe02-4868-b589-702008f84d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:45:46.558175 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1140/2222] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:45:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:45:55.167935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=43 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:45:56.544481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=22 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:46:09.796996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02135fb1-68fd-4c19-8d20-9d45ae64102a None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:46:09.797516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02135fb1-68fd-4c19-8d20-9d45ae64102a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:46:09.797621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02135fb1-68fd-4c19-8d20-9d45ae64102a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:46:09.814292 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-02135fb1-68fd-4c19-8d20-9d45ae64102a None None] MFA Rules not processed for user `a892b2511463404c818c63e870a23389`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:46:09.815531 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-02135fb1-68fd-4c19-8d20-9d45ae64102a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:46:09.872369 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1083/2223] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:46:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:46:19.865396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=25 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:46:36.476041 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81467f53-b05d-4304-8866-f0ad81a3f134 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:46:36.476546 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81467f53-b05d-4304-8866-f0ad81a3f134 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:46:36.476648 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81467f53-b05d-4304-8866-f0ad81a3f134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:46:36.494945 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-81467f53-b05d-4304-8866-f0ad81a3f134 None None] MFA Rules not processed for user `4a6b3bd40ad744908a05df7410cace52`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:46:36.496179 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-81467f53-b05d-4304-8866-f0ad81a3f134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:46:36.550934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1141/2224] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:46:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:46:43.603899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586e8e-8aca-4c5b-855b-466cc6423836 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:46:43.604426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586e8e-8aca-4c5b-855b-466cc6423836 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:46:43.604520 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586e8e-8aca-4c5b-855b-466cc6423836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:46:43.622778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b6586e8e-8aca-4c5b-855b-466cc6423836 None None] MFA Rules not processed for user `d9e5bc258de54fd6b3811401783a7408`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:46:43.623990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b6586e8e-8aca-4c5b-855b-466cc6423836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:46:43.678099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1084/2225] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:46:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:46:46.544142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=25 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:46:52.413061 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81467f53-b05d-4304-8866-f0ad81a3f134 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:46:52.423335 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81467f53-b05d-4304-8866-f0ad81a3f134 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:46:52.426905 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbfac2-5ad1-4dd0-b03d-3fd71991496b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:46:52.427543 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbfac2-5ad1-4dd0-b03d-3fd71991496b service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:46:52.427828 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbfac2-5ad1-4dd0-b03d-3fd71991496b service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:46:52.428107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbfac2-5ad1-4dd0-b03d-3fd71991496b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:46:52.429920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-53fbfac2-5ad1-4dd0-b03d-3fd71991496b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:46:52.501586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fbfac2-5ad1-4dd0-b03d-3fd71991496b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:46:52.503334 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fbfac2-5ad1-4dd0-b03d-3fd71991496b service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:46:52.514802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1142/2226] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:46:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:46:53.671787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=25 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:46:53.698707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6586e8e-8aca-4c5b-855b-466cc6423836 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:46:53.700076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b6586e8e-8aca-4c5b-855b-466cc6423836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:46:53.712526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6586e8e-8aca-4c5b-855b-466cc6423836 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:46:53.716199 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dcc874-5af3-4dc6-938a-7f70fb9aba4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:46:53.716852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcc874-5af3-4dc6-938a-7f70fb9aba4b service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:46:53.717135 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcc874-5af3-4dc6-938a-7f70fb9aba4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:46:53.717413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcc874-5af3-4dc6-938a-7f70fb9aba4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:46:53.719373 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d7dcc874-5af3-4dc6-938a-7f70fb9aba4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:46:53.733089 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dcc874-5af3-4dc6-938a-7f70fb9aba4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:46:53.734558 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dcc874-5af3-4dc6-938a-7f70fb9aba4b service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:46:53.746001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1085/2227] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:46:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:46:54.351531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3156f-171e-4cbf-be6c-a32a2517dbf9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:46:54.352047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3156f-171e-4cbf-be6c-a32a2517dbf9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:46:54.352150 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3156f-171e-4cbf-be6c-a32a2517dbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:46:54.371327 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-01d3156f-171e-4cbf-be6c-a32a2517dbf9 None None] MFA Rules not processed for user `1a50af3c7d004e78a99875e1f36a9508`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:46:54.372534 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-01d3156f-171e-4cbf-be6c-a32a2517dbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:46:54.379750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1143/2228] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:46:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:47:03.740639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:47:04.365169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=37 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:47:05.627700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f6957-bc33-4106-b5f0-32b9299130ba None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:05.628237 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f6957-bc33-4106-b5f0-32b9299130ba None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:05.628335 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f6957-bc33-4106-b5f0-32b9299130ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:05.648472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b13f6957-bc33-4106-b5f0-32b9299130ba None None] MFA Rules not processed for user `2ad8ebe54a1d42eda9b23108f92a1d23`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:47:05.650343 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b13f6957-bc33-4106-b5f0-32b9299130ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:05.715129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1086/2229] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:47:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:47:10.851861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3156f-171e-4cbf-be6c-a32a2517dbf9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:10.853952 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-01d3156f-171e-4cbf-be6c-a32a2517dbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:10.871816 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3156f-171e-4cbf-be6c-a32a2517dbf9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:10.877869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c6c0d-99c5-4ea8-a4e7-c0e8623d643d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:10.878548 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c6c0d-99c5-4ea8-a4e7-c0e8623d643d service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:10.878834 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c6c0d-99c5-4ea8-a4e7-c0e8623d643d service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:10.879112 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c6c0d-99c5-4ea8-a4e7-c0e8623d643d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:10.880913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-813c6c0d-99c5-4ea8-a4e7-c0e8623d643d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:10.891868 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c6c0d-99c5-4ea8-a4e7-c0e8623d643d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:10.893426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c6c0d-99c5-4ea8-a4e7-c0e8623d643d service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:10.903083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1144/2230] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:47:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:47:15.707244 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=25 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:47:17.366339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f6957-bc33-4106-b5f0-32b9299130ba None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:17.367857 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b13f6957-bc33-4106-b5f0-32b9299130ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:17.380014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f6957-bc33-4106-b5f0-32b9299130ba None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:17.384295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd3f9d-2111-4919-b96b-faa4ae9bc815 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:17.384966 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd3f9d-2111-4919-b96b-faa4ae9bc815 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:17.385220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd3f9d-2111-4919-b96b-faa4ae9bc815 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:17.385498 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd3f9d-2111-4919-b96b-faa4ae9bc815 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:17.387234 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e4fd3f9d-2111-4919-b96b-faa4ae9bc815 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:17.398062 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd3f9d-2111-4919-b96b-faa4ae9bc815 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:17.400450 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd3f9d-2111-4919-b96b-faa4ae9bc815 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:17.409282 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1087/2231] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:47:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:47:20.898401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=6 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:47:22.824906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c6c0d-99c5-4ea8-a4e7-c0e8623d643d service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:22.836101 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c6c0d-99c5-4ea8-a4e7-c0e8623d643d service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:22.842060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4491d-b5ce-45d3-9c51-2b08e4be9162 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:22.842815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4491d-b5ce-45d3-9c51-2b08e4be9162 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:22.843099 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4491d-b5ce-45d3-9c51-2b08e4be9162 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:22.843379 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4491d-b5ce-45d3-9c51-2b08e4be9162 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:22.845169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e5d4491d-b5ce-45d3-9c51-2b08e4be9162 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:22.859171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4491d-b5ce-45d3-9c51-2b08e4be9162 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:22.860790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4491d-b5ce-45d3-9c51-2b08e4be9162 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:22.872053 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1145/2232] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:47:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:47:23.991214 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd3f9d-2111-4919-b96b-faa4ae9bc815 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:24.001065 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd3f9d-2111-4919-b96b-faa4ae9bc815 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:24.005560 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339684c-3ddc-4b4a-b02a-95553046541f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:24.006260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339684c-3ddc-4b4a-b02a-95553046541f service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:24.006630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339684c-3ddc-4b4a-b02a-95553046541f service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:24.006915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339684c-3ddc-4b4a-b02a-95553046541f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:24.009411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c339684c-3ddc-4b4a-b02a-95553046541f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:24.020339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339684c-3ddc-4b4a-b02a-95553046541f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:24.021897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339684c-3ddc-4b4a-b02a-95553046541f service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:24.030322 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1088/2233] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:47:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:47:29.264174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4491d-b5ce-45d3-9c51-2b08e4be9162 service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:29.273467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4491d-b5ce-45d3-9c51-2b08e4be9162 service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:29.277109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbaa339-9698-4441-ad5f-c8b3fe1fa748 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:29.277762 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaa339-9698-4441-ad5f-c8b3fe1fa748 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:29.278081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaa339-9698-4441-ad5f-c8b3fe1fa748 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:29.278386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaa339-9698-4441-ad5f-c8b3fe1fa748 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:29.280132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6cbaa339-9698-4441-ad5f-c8b3fe1fa748 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:29.291582 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbaa339-9698-4441-ad5f-c8b3fe1fa748 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:29.293006 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbaa339-9698-4441-ad5f-c8b3fe1fa748 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:29.302160 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1146/2234] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:47:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:47:34.026580 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=12 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:47:34.421830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4560d-7711-410a-94f3-eac2184a39b9 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:34.422164 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4560d-7711-410a-94f3-eac2184a39b9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:34.422449 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4560d-7711-410a-94f3-eac2184a39b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:34.440294 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-cdc4560d-7711-410a-94f3-eac2184a39b9 None None] MFA Rules not processed for user `a892b2511463404c818c63e870a23389`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:47:34.441503 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cdc4560d-7711-410a-94f3-eac2184a39b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:34.449240 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1089/2235] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:47:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:47:34.750845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbaa339-9698-4441-ad5f-c8b3fe1fa748 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:34.760350 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbaa339-9698-4441-ad5f-c8b3fe1fa748 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:34.763996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7779a-6599-4b68-a821-829b18d2fbb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:34.764637 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7779a-6599-4b68-a821-829b18d2fbb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:34.764922 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7779a-6599-4b68-a821-829b18d2fbb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:34.765201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7779a-6599-4b68-a821-829b18d2fbb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:34.766991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ecc7779a-6599-4b68-a821-829b18d2fbb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:34.777145 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7779a-6599-4b68-a821-829b18d2fbb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:34.778653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7779a-6599-4b68-a821-829b18d2fbb1 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:34.786552 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1147/2236] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:47:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:47:35.101093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4560d-7711-410a-94f3-eac2184a39b9 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:35.102865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cdc4560d-7711-410a-94f3-eac2184a39b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:35.175428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4560d-7711-410a-94f3-eac2184a39b9 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:35.180539 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03aaf5a-1d39-4aa9-a243-cc9a30c4e4a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1308deea1ea4629a1ac22d4b903ad4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:35.181181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03aaf5a-1d39-4aa9-a243-cc9a30c4e4a2 service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:35.181461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03aaf5a-1d39-4aa9-a243-cc9a30c4e4a2 service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:35.181731 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03aaf5a-1d39-4aa9-a243-cc9a30c4e4a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:35.183526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e03aaf5a-1d39-4aa9-a243-cc9a30c4e4a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:35.194590 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03aaf5a-1d39-4aa9-a243-cc9a30c4e4a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:35.196144 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03aaf5a-1d39-4aa9-a243-cc9a30c4e4a2 service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:35.205764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1090/2237] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:47:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:47:44.783097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=18 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:47:45.201606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=35 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:47:47.014134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7779a-6599-4b68-a821-829b18d2fbb1 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:47.015459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ecc7779a-6599-4b68-a821-829b18d2fbb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:47.031786 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7779a-6599-4b68-a821-829b18d2fbb1 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:47.037332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48bea4-41a0-4f15-9bcc-d779608eab99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:47.037979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48bea4-41a0-4f15-9bcc-d779608eab99 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:47.038292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48bea4-41a0-4f15-9bcc-d779608eab99 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:47.038574 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48bea4-41a0-4f15-9bcc-d779608eab99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:47.040305 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9f48bea4-41a0-4f15-9bcc-d779608eab99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:47.051832 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48bea4-41a0-4f15-9bcc-d779608eab99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:47.053461 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48bea4-41a0-4f15-9bcc-d779608eab99 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:47.063278 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1148/2238] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:47:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:47:47.202388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03aaf5a-1d39-4aa9-a243-cc9a30c4e4a2 service placement] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:47.218480 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03aaf5a-1d39-4aa9-a243-cc9a30c4e4a2 service placement] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:47.224509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a192e2-ff4f-4a3d-9943-bed4a82709aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:47.225579 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a192e2-ff4f-4a3d-9943-bed4a82709aa service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:47.226006 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a192e2-ff4f-4a3d-9943-bed4a82709aa service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:47.226510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a192e2-ff4f-4a3d-9943-bed4a82709aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:47.228558 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-98a192e2-ff4f-4a3d-9943-bed4a82709aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:47.246216 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a192e2-ff4f-4a3d-9943-bed4a82709aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:47.247902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a192e2-ff4f-4a3d-9943-bed4a82709aa service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:47.261018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1091/2239] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:47:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:47:52.842693 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48bea4-41a0-4f15-9bcc-d779608eab99 service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:52.854722 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48bea4-41a0-4f15-9bcc-d779608eab99 service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:52.860374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c65e0-ec5b-4a54-951d-decaebb17111 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:52.861024 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c65e0-ec5b-4a54-951d-decaebb17111 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:52.861292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c65e0-ec5b-4a54-951d-decaebb17111 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:52.861572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c65e0-ec5b-4a54-951d-decaebb17111 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:52.863391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e35c65e0-ec5b-4a54-951d-decaebb17111 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:52.876423 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c65e0-ec5b-4a54-951d-decaebb17111 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:52.878382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c65e0-ec5b-4a54-951d-decaebb17111 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:52.889519 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1149/2240] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:47:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:47:55.415694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a192e2-ff4f-4a3d-9943-bed4a82709aa service nova] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:55.425619 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a192e2-ff4f-4a3d-9943-bed4a82709aa service nova] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:55.429232 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b87e3a-2700-4c48-b9fb-b78a0d3fe643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:55.429860 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b87e3a-2700-4c48-b9fb-b78a0d3fe643 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:55.430252 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b87e3a-2700-4c48-b9fb-b78a0d3fe643 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:55.430531 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b87e3a-2700-4c48-b9fb-b78a0d3fe643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:55.432309 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-70b87e3a-2700-4c48-b9fb-b78a0d3fe643 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:55.443776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b87e3a-2700-4c48-b9fb-b78a0d3fe643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:55.445189 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b87e3a-2700-4c48-b9fb-b78a0d3fe643 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:55.455224 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1092/2241] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:47:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:47:58.784592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c65e0-ec5b-4a54-951d-decaebb17111 service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:58.786096 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e35c65e0-ec5b-4a54-951d-decaebb17111 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:58.799754 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c65e0-ec5b-4a54-951d-decaebb17111 service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:58.804341 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445d1b2-30d9-4f4b-9231-1ec36f0aac8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:58.805218 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445d1b2-30d9-4f4b-9231-1ec36f0aac8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:58.805499 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445d1b2-30d9-4f4b-9231-1ec36f0aac8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:58.805773 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445d1b2-30d9-4f4b-9231-1ec36f0aac8d admin admin] PATH_INFO: `/v3/users/4a6b3bd40ad744908a05df7410cace52` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:58.807514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445d1b2-30d9-4f4b-9231-1ec36f0aac8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a6b3bd40ad744908a05df7410cace52)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:58.809437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445d1b2-30d9-4f4b-9231-1ec36f0aac8d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:58.859272 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4445d1b2-30d9-4f4b-9231-1ec36f0aac8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a6b3bd40ad744908a05df7410cace52'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:47:58.859931 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4445d1b2-30d9-4f4b-9231-1ec36f0aac8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a6b3bd40ad744908a05df7410cace52'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:47:58.860536 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4445d1b2-30d9-4f4b-9231-1ec36f0aac8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a6b3bd40ad744908a05df7410cace52'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:47:58.872988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4445d1b2-30d9-4f4b-9231-1ec36f0aac8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a6b3bd40ad744908a05df7410cace52'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:47:58.880540 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4445d1b2-30d9-4f4b-9231-1ec36f0aac8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a6b3bd40ad744908a05df7410cace52'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:47:58.895206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1150/2242] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:47:58 2022] DELETE /identity/v3/users/4a6b3bd40ad744908a05df7410cace52 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:47:59.068944 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b87e3a-2700-4c48-b9fb-b78a0d3fe643 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:59.083019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b87e3a-2700-4c48-b9fb-b78a0d3fe643 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:59.141018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fda0a-a813-4491-a85b-f349a3f51247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:59.141881 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fda0a-a813-4491-a85b-f349a3f51247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:59.142199 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fda0a-a813-4491-a85b-f349a3f51247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:59.142481 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fda0a-a813-4491-a85b-f349a3f51247 admin admin] PATH_INFO: `/v3/projects/026f214eb4ee4a9c807b928710c93db5` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:59.143664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96fda0a-a813-4491-a85b-f349a3f51247 admin admin] RBAC: Authorizing `identity:delete_project(project_id=026f214eb4ee4a9c807b928710c93db5)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:59.145561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96fda0a-a813-4491-a85b-f349a3f51247 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:59.190454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e96fda0a-a813-4491-a85b-f349a3f51247 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '026f214eb4ee4a9c807b928710c93db5', 'request_id': 'req-e96fda0a-a813-4491-a85b-f349a3f51247'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:47:59.195918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e96fda0a-a813-4491-a85b-f349a3f51247 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '026f214eb4ee4a9c807b928710c93db5', 'request_id': 'req-e96fda0a-a813-4491-a85b-f349a3f51247'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:47:59.203540 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e96fda0a-a813-4491-a85b-f349a3f51247 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '026f214eb4ee4a9c807b928710c93db5', 'request_id': 'req-e96fda0a-a813-4491-a85b-f349a3f51247'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:47:59.214273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e96fda0a-a813-4491-a85b-f349a3f51247 admin admin] The token cache is being invalidate because project 026f214eb4ee4a9c807b928710c93db5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:47:59.214561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e96fda0a-a813-4491-a85b-f349a3f51247 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:47:59.215188 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e96fda0a-a813-4491-a85b-f349a3f51247 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:47:59.216764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1093/2243] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:47:59 2022] DELETE /identity/v3/projects/026f214eb4ee4a9c807b928710c93db5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:47:59.221766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330b4ec-7263-4b29-bd71-61e92eaecc38 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:59.222087 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330b4ec-7263-4b29-bd71-61e92eaecc38 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:59.222411 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330b4ec-7263-4b29-bd71-61e92eaecc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:59.241716 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1330b4ec-7263-4b29-bd71-61e92eaecc38 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:47:59.243002 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1330b4ec-7263-4b29-bd71-61e92eaecc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:59.301677 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1151/2244] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:47:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:47:59.310044 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce61a2-be62-4c14-85a9-5672298d5b1b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:59.310340 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce61a2-be62-4c14-85a9-5672298d5b1b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:59.310619 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce61a2-be62-4c14-85a9-5672298d5b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:59.328399 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3dce61a2-be62-4c14-85a9-5672298d5b1b None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:47:59.329604 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3dce61a2-be62-4c14-85a9-5672298d5b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:59.337516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1094/2245] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:47:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:47:59.341679 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330b4ec-7263-4b29-bd71-61e92eaecc38 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:59.351355 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330b4ec-7263-4b29-bd71-61e92eaecc38 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:59.356512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786f4cc-6d41-4dc3-88f4-bedae853680f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:59.357159 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786f4cc-6d41-4dc3-88f4-bedae853680f admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:59.357449 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786f4cc-6d41-4dc3-88f4-bedae853680f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:59.357726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786f4cc-6d41-4dc3-88f4-bedae853680f admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:59.358503 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786f4cc-6d41-4dc3-88f4-bedae853680f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:47:59.358812 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786f4cc-6d41-4dc3-88f4-bedae853680f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:59.360413 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786f4cc-6d41-4dc3-88f4-bedae853680f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:59.372345 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1152/2246] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:47:59.376509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce61a2-be62-4c14-85a9-5672298d5b1b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:59.385793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce61a2-be62-4c14-85a9-5672298d5b1b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:59.390477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fbe763-3abe-4232-89f4-71b53394946b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:59.391113 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbe763-3abe-4232-89f4-71b53394946b admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:59.391395 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbe763-3abe-4232-89f4-71b53394946b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:59.391671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbe763-3abe-4232-89f4-71b53394946b admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:59.392353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fbe763-3abe-4232-89f4-71b53394946b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:59.394260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fbe763-3abe-4232-89f4-71b53394946b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:59.419751 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1095/2247] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:47:59 2022] POST /identity/v3/projects => generated 403 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:47:59.423790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786f4cc-6d41-4dc3-88f4-bedae853680f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:59.434208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786f4cc-6d41-4dc3-88f4-bedae853680f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:59.462810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:47:59.491956 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aba00a-3390-4b2d-a712-cbc7d66317d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:59.492619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aba00a-3390-4b2d-a712-cbc7d66317d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:59.492927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aba00a-3390-4b2d-a712-cbc7d66317d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:59.493242 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aba00a-3390-4b2d-a712-cbc7d66317d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:59.494030 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aba00a-3390-4b2d-a712-cbc7d66317d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:59.496588 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aba00a-3390-4b2d-a712-cbc7d66317d5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:59.516968 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1153/2248] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:47:59 2022] POST /identity/v3/users => generated 302 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:47:59.520945 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fbe763-3abe-4232-89f4-71b53394946b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:59.531698 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fbe763-3abe-4232-89f4-71b53394946b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:59.536633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a13c6-f6f6-411f-9d3d-aa0ee7ec2436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:59.541789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a13c6-f6f6-411f-9d3d-aa0ee7ec2436 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:59.542111 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a13c6-f6f6-411f-9d3d-aa0ee7ec2436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:59.542879 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a13c6-f6f6-411f-9d3d-aa0ee7ec2436 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:59.543618 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a13c6-f6f6-411f-9d3d-aa0ee7ec2436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:59.545225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a13c6-f6f6-411f-9d3d-aa0ee7ec2436 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:59.558843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1096/2249] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:47:59.563192 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aba00a-3390-4b2d-a712-cbc7d66317d5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:59.574550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aba00a-3390-4b2d-a712-cbc7d66317d5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:59.581231 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d5950-35a7-4739-8078-73b4dc562d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:59.582017 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d5950-35a7-4739-8078-73b4dc562d9f admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:59.582407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d5950-35a7-4739-8078-73b4dc562d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:59.582749 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d5950-35a7-4739-8078-73b4dc562d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:59.583596 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d5950-35a7-4739-8078-73b4dc562d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:59.585590 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d5950-35a7-4739-8078-73b4dc562d9f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:59.598237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1154/2250] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:47:59.603128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a13c6-f6f6-411f-9d3d-aa0ee7ec2436 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:59.616638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a13c6-f6f6-411f-9d3d-aa0ee7ec2436 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:59.622723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1e022-c8b7-4bad-aba3-f6bd7aa670e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:59.623615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1e022-c8b7-4bad-aba3-f6bd7aa670e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:59.623889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1e022-c8b7-4bad-aba3-f6bd7aa670e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:59.624327 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1e022-c8b7-4bad-aba3-f6bd7aa670e1 admin admin] PATH_INFO: `/v3/projects/e1a574fbcdeb49d7b3258d2bb8a00bf3/users/842221827e12427c8ab276f52bcc9933/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:59.645696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1e022-c8b7-4bad-aba3-f6bd7aa670e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a574fbcdeb49d7b3258d2bb8a00bf3, user_id=842221827e12427c8ab276f52bcc9933, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:59.648410 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1e022-c8b7-4bad-aba3-f6bd7aa670e1 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:59.658573 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:47:59.665365 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1097/2251] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:47:59 2022] PUT /identity/v3/projects/e1a574fbcdeb49d7b3258d2bb8a00bf3/users/842221827e12427c8ab276f52bcc9933/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:47:59.677104 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124f04c-1e3c-47aa-81fa-d2cb4c5c4aff None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:59.677496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124f04c-1e3c-47aa-81fa-d2cb4c5c4aff None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:59.677841 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124f04c-1e3c-47aa-81fa-d2cb4c5c4aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:59.705625 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-0124f04c-1e3c-47aa-81fa-d2cb4c5c4aff None None] MFA Rules not processed for user `842221827e12427c8ab276f52bcc9933`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:47:59.706880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0124f04c-1e3c-47aa-81fa-d2cb4c5c4aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:47:59.783414 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1155/2252] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:47:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:47:59.784465 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1e022-c8b7-4bad-aba3-f6bd7aa670e1 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:59.888120 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1e022-c8b7-4bad-aba3-f6bd7aa670e1 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:59.892870 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a62603-bf27-406f-ace8-4aba8d731baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:59.893523 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a62603-bf27-406f-ace8-4aba8d731baa admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:59.893801 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a62603-bf27-406f-ace8-4aba8d731baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:59.894109 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a62603-bf27-406f-ace8-4aba8d731baa admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:59.894810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a62603-bf27-406f-ace8-4aba8d731baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:47:59.896716 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a62603-bf27-406f-ace8-4aba8d731baa admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:47:59.926258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1098/2253] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:47:59 2022] POST /identity/v3/projects => generated 405 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:47:59.928440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124f04c-1e3c-47aa-81fa-d2cb4c5c4aff None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:47:59.939521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124f04c-1e3c-47aa-81fa-d2cb4c5c4aff None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:47:59.997671 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10adf65b-1f61-4d68-ae69-2566a4af2e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:47:59.998403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adf65b-1f61-4d68-ae69-2566a4af2e3e admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:47:59.998691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adf65b-1f61-4d68-ae69-2566a4af2e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:47:59.998971 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adf65b-1f61-4d68-ae69-2566a4af2e3e admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:47:59.999658 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10adf65b-1f61-4d68-ae69-2566a4af2e3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:00.001630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10adf65b-1f61-4d68-ae69-2566a4af2e3e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:00.022077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1156/2254] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:47:59 2022] POST /identity/v3/users => generated 309 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:00.026093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a62603-bf27-406f-ace8-4aba8d731baa admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:00.037163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a62603-bf27-406f-ace8-4aba8d731baa admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:00.042032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d81310-ec9c-4a94-8bab-4226d96aed0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:00.042719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d81310-ec9c-4a94-8bab-4226d96aed0d admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:00.043007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d81310-ec9c-4a94-8bab-4226d96aed0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:00.043285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d81310-ec9c-4a94-8bab-4226d96aed0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:00.044012 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d81310-ec9c-4a94-8bab-4226d96aed0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:00.045618 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d81310-ec9c-4a94-8bab-4226d96aed0d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:00.057226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1099/2255] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:00.061819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10adf65b-1f61-4d68-ae69-2566a4af2e3e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:00.071946 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10adf65b-1f61-4d68-ae69-2566a4af2e3e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:00.077607 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d3806-fd4a-4e20-b828-21e800cd68a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:00.078527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d3806-fd4a-4e20-b828-21e800cd68a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:00.078817 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d3806-fd4a-4e20-b828-21e800cd68a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:00.079094 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d3806-fd4a-4e20-b828-21e800cd68a3 admin admin] PATH_INFO: `/v3/projects/3fe9d883802d4c2cbfd5980208e63297/users/3b3d1f66ce3f4db08d435b71860f3ce4/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:00.094464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d3806-fd4a-4e20-b828-21e800cd68a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fe9d883802d4c2cbfd5980208e63297, user_id=3b3d1f66ce3f4db08d435b71860f3ce4, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:00.097136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d3806-fd4a-4e20-b828-21e800cd68a3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:00.110817 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1157/2256] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:00 2022] PUT /identity/v3/projects/3fe9d883802d4c2cbfd5980208e63297/users/3b3d1f66ce3f4db08d435b71860f3ce4/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:48:00.115569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d81310-ec9c-4a94-8bab-4226d96aed0d admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:00.126600 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d81310-ec9c-4a94-8bab-4226d96aed0d admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:00.129774 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d3806-fd4a-4e20-b828-21e800cd68a3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:00.131533 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e14d3806-fd4a-4e20-b828-21e800cd68a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:00.143246 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:00.191776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276dc75-71a1-40bf-bc96-d721d281f0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:00.192435 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276dc75-71a1-40bf-bc96-d721d281f0eb admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:00.192713 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276dc75-71a1-40bf-bc96-d721d281f0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:00.193229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276dc75-71a1-40bf-bc96-d721d281f0eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:00.193818 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276dc75-71a1-40bf-bc96-d721d281f0eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:00.195442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276dc75-71a1-40bf-bc96-d721d281f0eb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:00.205330 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d3806-fd4a-4e20-b828-21e800cd68a3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:00.205640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1100/2257] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:00.209607 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276dc75-71a1-40bf-bc96-d721d281f0eb admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:00.213173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d15c6d-306d-43b5-9c0b-63c1188c20ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:00.214191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d15c6d-306d-43b5-9c0b-63c1188c20ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:00.214982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d15c6d-306d-43b5-9c0b-63c1188c20ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:00.215290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d15c6d-306d-43b5-9c0b-63c1188c20ed admin admin] PATH_INFO: `/v3/users/1a50af3c7d004e78a99875e1f36a9508` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:00.216499 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d15c6d-306d-43b5-9c0b-63c1188c20ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a50af3c7d004e78a99875e1f36a9508)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:00.218469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d15c6d-306d-43b5-9c0b-63c1188c20ed admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:00.220811 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276dc75-71a1-40bf-bc96-d721d281f0eb admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:00.225542 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce33d89-24a6-42bf-bb36-1ffa5da8d146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:00.226442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce33d89-24a6-42bf-bb36-1ffa5da8d146 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:00.226723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce33d89-24a6-42bf-bb36-1ffa5da8d146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:00.226998 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce33d89-24a6-42bf-bb36-1ffa5da8d146 admin admin] PATH_INFO: `/v3/projects/3fe9d883802d4c2cbfd5980208e63297/users/3b3d1f66ce3f4db08d435b71860f3ce4/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:00.229018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce33d89-24a6-42bf-bb36-1ffa5da8d146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fe9d883802d4c2cbfd5980208e63297, user_id=3b3d1f66ce3f4db08d435b71860f3ce4, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:00.231755 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce33d89-24a6-42bf-bb36-1ffa5da8d146 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:00.242777 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1101/2258] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:00 2022] PUT /identity/v3/projects/3fe9d883802d4c2cbfd5980208e63297/users/3b3d1f66ce3f4db08d435b71860f3ce4/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:48:00.254848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c7f05-3e7d-4074-b8ed-2f88e6b80b02 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:00.255187 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c7f05-3e7d-4074-b8ed-2f88e6b80b02 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:00.255807 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c7f05-3e7d-4074-b8ed-2f88e6b80b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:00.273024 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-68d15c6d-306d-43b5-9c0b-63c1188c20ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a50af3c7d004e78a99875e1f36a9508'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:00.273681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-68d15c6d-306d-43b5-9c0b-63c1188c20ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a50af3c7d004e78a99875e1f36a9508'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:00.274324 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-68d15c6d-306d-43b5-9c0b-63c1188c20ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a50af3c7d004e78a99875e1f36a9508'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:00.284379 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-842c7f05-3e7d-4074-b8ed-2f88e6b80b02 None None] MFA Rules not processed for user `3b3d1f66ce3f4db08d435b71860f3ce4`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:00.285594 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-842c7f05-3e7d-4074-b8ed-2f88e6b80b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:00.287501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-68d15c6d-306d-43b5-9c0b-63c1188c20ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a50af3c7d004e78a99875e1f36a9508'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:00.294642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-68d15c6d-306d-43b5-9c0b-63c1188c20ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a50af3c7d004e78a99875e1f36a9508'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:00.308932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1158/2259] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:48:00 2022] DELETE /identity/v3/users/1a50af3c7d004e78a99875e1f36a9508 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:48:00.333813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:00.360124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1102/2260] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:00 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:48:00.371305 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d15c6d-306d-43b5-9c0b-63c1188c20ed admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:00.372727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-68d15c6d-306d-43b5-9c0b-63c1188c20ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:00.431785 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d15c6d-306d-43b5-9c0b-63c1188c20ed admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:00.436064 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cad034-8b99-494e-b4b2-dbb285140fa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:00.436721 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cad034-8b99-494e-b4b2-dbb285140fa9 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:00.437007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cad034-8b99-494e-b4b2-dbb285140fa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:00.437313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cad034-8b99-494e-b4b2-dbb285140fa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:00.490606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:00.502355 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c7f05-3e7d-4074-b8ed-2f88e6b80b02 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:00.504919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-842c7f05-3e7d-4074-b8ed-2f88e6b80b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:00.526904 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cad034-8b99-494e-b4b2-dbb285140fa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:00.528378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cad034-8b99-494e-b4b2-dbb285140fa9 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:00.547386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1159/2261] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:00 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:00.591726 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c7f05-3e7d-4074-b8ed-2f88e6b80b02 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:00.663337 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad7bbf-6fb1-454a-a4b1-2581f8efcbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:00.664350 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad7bbf-6fb1-454a-a4b1-2581f8efcbef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:00.664633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad7bbf-6fb1-454a-a4b1-2581f8efcbef admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:00.665091 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad7bbf-6fb1-454a-a4b1-2581f8efcbef admin admin] PATH_INFO: `/v3/projects/adf5aa6b70414b8f81904a32fa4fa1ab` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:00.666402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad7bbf-6fb1-454a-a4b1-2581f8efcbef admin admin] RBAC: Authorizing `identity:delete_project(project_id=adf5aa6b70414b8f81904a32fa4fa1ab)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:00.668914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad7bbf-6fb1-454a-a4b1-2581f8efcbef admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:00.721458 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cad034-8b99-494e-b4b2-dbb285140fa9 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:00.722992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-28cad034-8b99-494e-b4b2-dbb285140fa9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:00.737877 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cad034-8b99-494e-b4b2-dbb285140fa9 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:00.739704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1aad7bbf-6fb1-454a-a4b1-2581f8efcbef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adf5aa6b70414b8f81904a32fa4fa1ab', 'request_id': 'req-1aad7bbf-6fb1-454a-a4b1-2581f8efcbef'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:00.746791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1aad7bbf-6fb1-454a-a4b1-2581f8efcbef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adf5aa6b70414b8f81904a32fa4fa1ab', 'request_id': 'req-1aad7bbf-6fb1-454a-a4b1-2581f8efcbef'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:00.757697 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1aad7bbf-6fb1-454a-a4b1-2581f8efcbef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adf5aa6b70414b8f81904a32fa4fa1ab', 'request_id': 'req-1aad7bbf-6fb1-454a-a4b1-2581f8efcbef'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:00.773984 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1aad7bbf-6fb1-454a-a4b1-2581f8efcbef admin admin] The token cache is being invalidate because project adf5aa6b70414b8f81904a32fa4fa1ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:48:00.774615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1aad7bbf-6fb1-454a-a4b1-2581f8efcbef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:00.775414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1aad7bbf-6fb1-454a-a4b1-2581f8efcbef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:00.782631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1103/2262] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:48:00 2022] DELETE /identity/v3/projects/adf5aa6b70414b8f81904a32fa4fa1ab => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:48:00.784305 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4acab5-a418-43bc-a55a-784a808334f0 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:00.784900 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4acab5-a418-43bc-a55a-784a808334f0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:00.785918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4acab5-a418-43bc-a55a-784a808334f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:00.808054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9a4acab5-a418-43bc-a55a-784a808334f0 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:00.809279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9a4acab5-a418-43bc-a55a-784a808334f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:00.811261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524058f-b12c-4662-af9a-9f950fd59d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:00.813419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524058f-b12c-4662-af9a-9f950fd59d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:00.813419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524058f-b12c-4662-af9a-9f950fd59d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:00.813419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524058f-b12c-4662-af9a-9f950fd59d13 admin admin] PATH_INFO: `/v3/users/842221827e12427c8ab276f52bcc9933` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:00.814962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524058f-b12c-4662-af9a-9f950fd59d13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=842221827e12427c8ab276f52bcc9933)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:00.817178 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524058f-b12c-4662-af9a-9f950fd59d13 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:00.818427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1104/2263] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:00.826152 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656fad4-da6f-4065-a1b7-f29d098e772f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:00.826445 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656fad4-da6f-4065-a1b7-f29d098e772f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:00.826728 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656fad4-da6f-4065-a1b7-f29d098e772f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:00.846858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e656fad4-da6f-4065-a1b7-f29d098e772f None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:00.848472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e656fad4-da6f-4065-a1b7-f29d098e772f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:00.863344 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1105/2264] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:48:00.867746 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656fad4-da6f-4065-a1b7-f29d098e772f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:00.879800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656fad4-da6f-4065-a1b7-f29d098e772f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:00.887264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a8da9-1ca5-4eac-8401-e89f041d1210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:00.888121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a8da9-1ca5-4eac-8401-e89f041d1210 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:00.888541 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a8da9-1ca5-4eac-8401-e89f041d1210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:00.888826 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a8da9-1ca5-4eac-8401-e89f041d1210 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:00.889623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a8da9-1ca5-4eac-8401-e89f041d1210 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:48:00.890342 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a8da9-1ca5-4eac-8401-e89f041d1210 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:00.892226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a8da9-1ca5-4eac-8401-e89f041d1210 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:00.894128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:00.896507 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b524058f-b12c-4662-af9a-9f950fd59d13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '842221827e12427c8ab276f52bcc9933'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:00.898747 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b524058f-b12c-4662-af9a-9f950fd59d13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '842221827e12427c8ab276f52bcc9933'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:00.899876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b524058f-b12c-4662-af9a-9f950fd59d13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '842221827e12427c8ab276f52bcc9933'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:00.915272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1106/2265] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:48:00.919810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a8da9-1ca5-4eac-8401-e89f041d1210 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:00.922170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-af5a8da9-1ca5-4eac-8401-e89f041d1210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:00.923559 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b524058f-b12c-4662-af9a-9f950fd59d13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '842221827e12427c8ab276f52bcc9933'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:00.933600 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b524058f-b12c-4662-af9a-9f950fd59d13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '842221827e12427c8ab276f52bcc9933'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:00.938330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a8da9-1ca5-4eac-8401-e89f041d1210 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:00.943950 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2b018-6f40-4837-9cbc-7f403b6b7f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:00.944617 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2b018-6f40-4837-9cbc-7f403b6b7f81 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:00.944910 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2b018-6f40-4837-9cbc-7f403b6b7f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:00.945171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2b018-6f40-4837-9cbc-7f403b6b7f81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:00.945887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2b018-6f40-4837-9cbc-7f403b6b7f81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:00.948028 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2b018-6f40-4837-9cbc-7f403b6b7f81 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:00.951802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1160/2266] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:48:00 2022] DELETE /identity/v3/users/842221827e12427c8ab276f52bcc9933 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:00.956328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200a92c-97cf-4d5a-bdf1-4cba9c400c04 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:00.956613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200a92c-97cf-4d5a-bdf1-4cba9c400c04 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:00.956887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200a92c-97cf-4d5a-bdf1-4cba9c400c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:00.975791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b200a92c-97cf-4d5a-bdf1-4cba9c400c04 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:00.977057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b200a92c-97cf-4d5a-bdf1-4cba9c400c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:00.989332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1107/2267] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:48:00 2022] POST /identity/v3/projects => generated 391 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:00.990695 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2b018-6f40-4837-9cbc-7f403b6b7f81 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:01.032430 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:01.050715 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1161/2268] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:01.062236 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200a92c-97cf-4d5a-bdf1-4cba9c400c04 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:01.064115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b200a92c-97cf-4d5a-bdf1-4cba9c400c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:01.067971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2b018-6f40-4837-9cbc-7f403b6b7f81 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:01.075314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e55c4-7515-48b0-838f-662b721631a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:01.076081 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e55c4-7515-48b0-838f-662b721631a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:01.076348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e55c4-7515-48b0-838f-662b721631a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:01.076625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e55c4-7515-48b0-838f-662b721631a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:01.077316 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e55c4-7515-48b0-838f-662b721631a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:01.079671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e55c4-7515-48b0-838f-662b721631a3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:01.116650 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1108/2269] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:48:00 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:01.122010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e55c4-7515-48b0-838f-662b721631a3 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:01.132670 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e55c4-7515-48b0-838f-662b721631a3 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:01.138151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efc194-fd81-41b5-85c7-a7671df82d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:01.138823 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efc194-fd81-41b5-85c7-a7671df82d09 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:01.139211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efc194-fd81-41b5-85c7-a7671df82d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:01.139500 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efc194-fd81-41b5-85c7-a7671df82d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:01.140148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200a92c-97cf-4d5a-bdf1-4cba9c400c04 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:01.140287 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7efc194-fd81-41b5-85c7-a7671df82d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:01.142406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7efc194-fd81-41b5-85c7-a7671df82d09 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:01.156638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1109/2270] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:48:01.157895 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efc194-fd81-41b5-85c7-a7671df82d09 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:01.168376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efc194-fd81-41b5-85c7-a7671df82d09 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:01.173147 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6bf24-f4a4-4d7d-931c-1b2f57c796ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:01.173908 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6bf24-f4a4-4d7d-931c-1b2f57c796ce admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:01.174344 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6bf24-f4a4-4d7d-931c-1b2f57c796ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:01.174731 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6bf24-f4a4-4d7d-931c-1b2f57c796ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:01.175555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6bf24-f4a4-4d7d-931c-1b2f57c796ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:01.177221 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6bf24-f4a4-4d7d-931c-1b2f57c796ce admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:01.188818 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1110/2271] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:48:01.192832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6bf24-f4a4-4d7d-931c-1b2f57c796ce admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:01.196034 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372bf1c-e6a4-4687-975a-401610ea3525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:01.196742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372bf1c-e6a4-4687-975a-401610ea3525 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:01.197028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372bf1c-e6a4-4687-975a-401610ea3525 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:01.197302 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372bf1c-e6a4-4687-975a-401610ea3525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:01.203413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6bf24-f4a4-4d7d-931c-1b2f57c796ce admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:01.208255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372bf1c-e6a4-4687-975a-401610ea3525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:01.208988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38e9c4-b839-4985-881b-22df9e93c478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:01.209777 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372bf1c-e6a4-4687-975a-401610ea3525 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:01.209901 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38e9c4-b839-4985-881b-22df9e93c478 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:01.210160 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38e9c4-b839-4985-881b-22df9e93c478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:01.210438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38e9c4-b839-4985-881b-22df9e93c478 admin admin] PATH_INFO: `/v3/projects/31e9a6cad6f54b8ea4080c864390afa9/users/b690cc866cf44fb99f209782bb7439ae/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:01.225904 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38e9c4-b839-4985-881b-22df9e93c478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e9a6cad6f54b8ea4080c864390afa9, user_id=b690cc866cf44fb99f209782bb7439ae, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:01.228601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38e9c4-b839-4985-881b-22df9e93c478 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:01.230486 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1162/2272] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:48:01.243721 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1111/2273] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:01 2022] PUT /identity/v3/projects/31e9a6cad6f54b8ea4080c864390afa9/users/b690cc866cf44fb99f209782bb7439ae/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:01.254718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a40f7e-9221-4cc0-a02e-33ca46d1c8ce None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:01.255009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a40f7e-9221-4cc0-a02e-33ca46d1c8ce None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:01.255319 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a40f7e-9221-4cc0-a02e-33ca46d1c8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:01.287805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-e4a40f7e-9221-4cc0-a02e-33ca46d1c8ce None None] MFA Rules not processed for user `b690cc866cf44fb99f209782bb7439ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:01.289417 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e4a40f7e-9221-4cc0-a02e-33ca46d1c8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:01.366727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1163/2274] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:01.379005 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38e9c4-b839-4985-881b-22df9e93c478 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:01.381281 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2e38e9c4-b839-4985-881b-22df9e93c478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:01.441271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38e9c4-b839-4985-881b-22df9e93c478 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:01.444908 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1994d-e5a6-452f-9408-1542c94c440f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:01.445582 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1994d-e5a6-452f-9408-1542c94c440f service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:01.445814 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1994d-e5a6-452f-9408-1542c94c440f service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:01.446169 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1994d-e5a6-452f-9408-1542c94c440f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:01.455360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1994d-e5a6-452f-9408-1542c94c440f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:01.456782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1994d-e5a6-452f-9408-1542c94c440f service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:01.466361 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1112/2275] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:01.503380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a40f7e-9221-4cc0-a02e-33ca46d1c8ce None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:01.505108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e4a40f7e-9221-4cc0-a02e-33ca46d1c8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:01.554863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:01.575186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a40f7e-9221-4cc0-a02e-33ca46d1c8ce None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:01.580678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6da17f-e41f-47ff-ba98-e56d34f113cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:01.581560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6da17f-e41f-47ff-ba98-e56d34f113cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:01.581835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6da17f-e41f-47ff-ba98-e56d34f113cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:01.582745 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6da17f-e41f-47ff-ba98-e56d34f113cf admin admin] PATH_INFO: `/v3/projects/e1a574fbcdeb49d7b3258d2bb8a00bf3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:01.583948 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6da17f-e41f-47ff-ba98-e56d34f113cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1a574fbcdeb49d7b3258d2bb8a00bf3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:01.586041 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6da17f-e41f-47ff-ba98-e56d34f113cf admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:01.637486 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-de6da17f-e41f-47ff-ba98-e56d34f113cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1a574fbcdeb49d7b3258d2bb8a00bf3', 'request_id': 'req-de6da17f-e41f-47ff-ba98-e56d34f113cf'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:01.645641 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-de6da17f-e41f-47ff-ba98-e56d34f113cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1a574fbcdeb49d7b3258d2bb8a00bf3', 'request_id': 'req-de6da17f-e41f-47ff-ba98-e56d34f113cf'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:01.652458 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-de6da17f-e41f-47ff-ba98-e56d34f113cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1a574fbcdeb49d7b3258d2bb8a00bf3', 'request_id': 'req-de6da17f-e41f-47ff-ba98-e56d34f113cf'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:01.663314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-de6da17f-e41f-47ff-ba98-e56d34f113cf admin admin] The token cache is being invalidate because project e1a574fbcdeb49d7b3258d2bb8a00bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:48:01.663634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-de6da17f-e41f-47ff-ba98-e56d34f113cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:01.664296 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-de6da17f-e41f-47ff-ba98-e56d34f113cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:01.666113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1164/2276] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:48:01 2022] DELETE /identity/v3/projects/e1a574fbcdeb49d7b3258d2bb8a00bf3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:01.670493 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1994d-e5a6-452f-9408-1542c94c440f service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:01.681735 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1994d-e5a6-452f-9408-1542c94c440f service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:01.694753 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:01.751273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c39b9-ec1c-4137-8734-c88f9fdfe11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:01.752476 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c39b9-ec1c-4137-8734-c88f9fdfe11f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:01.752872 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c39b9-ec1c-4137-8734-c88f9fdfe11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:01.753150 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c39b9-ec1c-4137-8734-c88f9fdfe11f admin admin] PATH_INFO: `/v3/users/3b3d1f66ce3f4db08d435b71860f3ce4` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:01.754381 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c39b9-ec1c-4137-8734-c88f9fdfe11f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b3d1f66ce3f4db08d435b71860f3ce4)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:01.756458 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c39b9-ec1c-4137-8734-c88f9fdfe11f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:01.809788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6e2c39b9-ec1c-4137-8734-c88f9fdfe11f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b3d1f66ce3f4db08d435b71860f3ce4'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:01.821768 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6e2c39b9-ec1c-4137-8734-c88f9fdfe11f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b3d1f66ce3f4db08d435b71860f3ce4'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:01.829198 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6e2c39b9-ec1c-4137-8734-c88f9fdfe11f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b3d1f66ce3f4db08d435b71860f3ce4'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:01.829867 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6e2c39b9-ec1c-4137-8734-c88f9fdfe11f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b3d1f66ce3f4db08d435b71860f3ce4'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:01.836886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6e2c39b9-ec1c-4137-8734-c88f9fdfe11f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b3d1f66ce3f4db08d435b71860f3ce4'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:01.843740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1113/2277] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:48:01 2022] DELETE /identity/v3/users/3b3d1f66ce3f4db08d435b71860f3ce4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:48:02.120472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6da17f-e41f-47ff-ba98-e56d34f113cf admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:02.123138 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-de6da17f-e41f-47ff-ba98-e56d34f113cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:02.136284 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6da17f-e41f-47ff-ba98-e56d34f113cf admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:02.202424 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9234b-7a1b-4708-96d9-c2931b53ffed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:02.203301 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9234b-7a1b-4708-96d9-c2931b53ffed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:02.203586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9234b-7a1b-4708-96d9-c2931b53ffed admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:02.203862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9234b-7a1b-4708-96d9-c2931b53ffed admin admin] PATH_INFO: `/v3/projects/3fe9d883802d4c2cbfd5980208e63297` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:02.205051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9234b-7a1b-4708-96d9-c2931b53ffed admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fe9d883802d4c2cbfd5980208e63297)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:02.207022 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9234b-7a1b-4708-96d9-c2931b53ffed admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:02.255762 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c1e9234b-7a1b-4708-96d9-c2931b53ffed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fe9d883802d4c2cbfd5980208e63297', 'request_id': 'req-c1e9234b-7a1b-4708-96d9-c2931b53ffed'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:02.264856 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c1e9234b-7a1b-4708-96d9-c2931b53ffed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fe9d883802d4c2cbfd5980208e63297', 'request_id': 'req-c1e9234b-7a1b-4708-96d9-c2931b53ffed'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:02.271188 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c1e9234b-7a1b-4708-96d9-c2931b53ffed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fe9d883802d4c2cbfd5980208e63297', 'request_id': 'req-c1e9234b-7a1b-4708-96d9-c2931b53ffed'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:02.283840 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c1e9234b-7a1b-4708-96d9-c2931b53ffed admin admin] The token cache is being invalidate because project 3fe9d883802d4c2cbfd5980208e63297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:48:02.284132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c1e9234b-7a1b-4708-96d9-c2931b53ffed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:02.284783 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c1e9234b-7a1b-4708-96d9-c2931b53ffed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:02.286572 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1165/2278] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:48:02 2022] DELETE /identity/v3/projects/3fe9d883802d4c2cbfd5980208e63297 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:02.291358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830cbdd-bfe2-4087-8c84-4799a52d6af0 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:02.291811 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830cbdd-bfe2-4087-8c84-4799a52d6af0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:02.291999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830cbdd-bfe2-4087-8c84-4799a52d6af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:02.309104 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e830cbdd-bfe2-4087-8c84-4799a52d6af0 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:02.310351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e830cbdd-bfe2-4087-8c84-4799a52d6af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:02.367949 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1114/2279] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:48:02.376611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8deb0df-ac19-4d0a-aa88-a76169bc5df2 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:02.377167 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8deb0df-ac19-4d0a-aa88-a76169bc5df2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:02.377480 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8deb0df-ac19-4d0a-aa88-a76169bc5df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:02.384800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:02.397736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b8deb0df-ac19-4d0a-aa88-a76169bc5df2 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:02.399179 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b8deb0df-ac19-4d0a-aa88-a76169bc5df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:02.407755 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1166/2280] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:02.411852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830cbdd-bfe2-4087-8c84-4799a52d6af0 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:02.421553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830cbdd-bfe2-4087-8c84-4799a52d6af0 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:02.426157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95d940-61d5-4a3f-8211-ad0c55dbaa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:02.426806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95d940-61d5-4a3f-8211-ad0c55dbaa53 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:02.427093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95d940-61d5-4a3f-8211-ad0c55dbaa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:02.427373 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95d940-61d5-4a3f-8211-ad0c55dbaa53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:02.428101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95d940-61d5-4a3f-8211-ad0c55dbaa53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:48:02.428409 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95d940-61d5-4a3f-8211-ad0c55dbaa53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:02.430051 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95d940-61d5-4a3f-8211-ad0c55dbaa53 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:02.441378 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1115/2281] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:48:02.445447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8deb0df-ac19-4d0a-aa88-a76169bc5df2 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:02.455226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8deb0df-ac19-4d0a-aa88-a76169bc5df2 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:02.461447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daaaf95-d792-4803-95ad-c73666532878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:02.462152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daaaf95-d792-4803-95ad-c73666532878 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:02.462440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daaaf95-d792-4803-95ad-c73666532878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:02.462722 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daaaf95-d792-4803-95ad-c73666532878 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:02.463420 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daaaf95-d792-4803-95ad-c73666532878 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:02.465923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daaaf95-d792-4803-95ad-c73666532878 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:02.492892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1167/2282] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:48:02 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:02.497596 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95d940-61d5-4a3f-8211-ad0c55dbaa53 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:02.507853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95d940-61d5-4a3f-8211-ad0c55dbaa53 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:02.528960 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:02.565814 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e062fc-dbc7-446a-bffa-9215555877d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:02.566511 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e062fc-dbc7-446a-bffa-9215555877d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:02.566798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e062fc-dbc7-446a-bffa-9215555877d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:02.567075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e062fc-dbc7-446a-bffa-9215555877d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:02.567765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e062fc-dbc7-446a-bffa-9215555877d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:02.569687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e062fc-dbc7-446a-bffa-9215555877d3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:02.590355 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1116/2283] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:48:02 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:02.594440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daaaf95-d792-4803-95ad-c73666532878 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:02.606727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daaaf95-d792-4803-95ad-c73666532878 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:02.611485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab315a-bbcb-4210-9b72-8be6f086e772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:02.612150 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab315a-bbcb-4210-9b72-8be6f086e772 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:02.612435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab315a-bbcb-4210-9b72-8be6f086e772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:02.612708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab315a-bbcb-4210-9b72-8be6f086e772 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:02.613438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab315a-bbcb-4210-9b72-8be6f086e772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:02.615114 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab315a-bbcb-4210-9b72-8be6f086e772 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:02.625448 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1168/2284] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:02.629499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e062fc-dbc7-446a-bffa-9215555877d3 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:02.639760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e062fc-dbc7-446a-bffa-9215555877d3 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:02.644625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc6451-992f-4a3a-a7fb-9ab9a30916e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:02.645306 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc6451-992f-4a3a-a7fb-9ab9a30916e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:02.645589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc6451-992f-4a3a-a7fb-9ab9a30916e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:02.645866 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc6451-992f-4a3a-a7fb-9ab9a30916e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:02.646621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc6451-992f-4a3a-a7fb-9ab9a30916e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:02.648205 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc6451-992f-4a3a-a7fb-9ab9a30916e4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:02.658472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1117/2285] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:02.662568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab315a-bbcb-4210-9b72-8be6f086e772 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:02.673141 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab315a-bbcb-4210-9b72-8be6f086e772 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:02.678621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34480d01-ae4d-4fce-95da-d5155f7db76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:02.679491 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34480d01-ae4d-4fce-95da-d5155f7db76a admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:02.679760 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34480d01-ae4d-4fce-95da-d5155f7db76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:02.680012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34480d01-ae4d-4fce-95da-d5155f7db76a admin admin] PATH_INFO: `/v3/projects/7266a9f98bec48919e1b2417e9792ca6/users/193d69cecd9e4207b25a26a684c75acd/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:02.694514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34480d01-ae4d-4fce-95da-d5155f7db76a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7266a9f98bec48919e1b2417e9792ca6, user_id=193d69cecd9e4207b25a26a684c75acd, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:02.697151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34480d01-ae4d-4fce-95da-d5155f7db76a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:02.710138 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1169/2286] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:02 2022] PUT /identity/v3/projects/7266a9f98bec48919e1b2417e9792ca6/users/193d69cecd9e4207b25a26a684c75acd/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:02.720864 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e72f88-7a38-4bdb-b3e2-4203e4e7f5c8 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:02.721157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e72f88-7a38-4bdb-b3e2-4203e4e7f5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:02.721434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e72f88-7a38-4bdb-b3e2-4203e4e7f5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:02.749229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-12e72f88-7a38-4bdb-b3e2-4203e4e7f5c8 None None] MFA Rules not processed for user `193d69cecd9e4207b25a26a684c75acd`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:02.750530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-12e72f88-7a38-4bdb-b3e2-4203e4e7f5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:02.834901 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1118/2287] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:02.840803 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34480d01-ae4d-4fce-95da-d5155f7db76a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:02.912680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34480d01-ae4d-4fce-95da-d5155f7db76a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:02.918519 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23250a6f-ddb1-48aa-b148-4167c3cf7670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:02.919246 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23250a6f-ddb1-48aa-b148-4167c3cf7670 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:02.919631 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23250a6f-ddb1-48aa-b148-4167c3cf7670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:02.919910 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23250a6f-ddb1-48aa-b148-4167c3cf7670 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:02.920592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23250a6f-ddb1-48aa-b148-4167c3cf7670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:02.922845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23250a6f-ddb1-48aa-b148-4167c3cf7670 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:02.950881 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1170/2288] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:48:02 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:02.952015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e72f88-7a38-4bdb-b3e2-4203e4e7f5c8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:02.962580 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e72f88-7a38-4bdb-b3e2-4203e4e7f5c8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:03.037058 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a552d-12e9-4626-9746-46b3cba931e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:03.037735 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a552d-12e9-4626-9746-46b3cba931e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:03.038065 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a552d-12e9-4626-9746-46b3cba931e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:03.038345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a552d-12e9-4626-9746-46b3cba931e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:03.039041 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a552d-12e9-4626-9746-46b3cba931e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:03.041116 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a552d-12e9-4626-9746-46b3cba931e0 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:03.060660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1119/2289] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:48:02 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:03.064610 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23250a6f-ddb1-48aa-b148-4167c3cf7670 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:03.075527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23250a6f-ddb1-48aa-b148-4167c3cf7670 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:03.080158 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e8de0-896d-450b-bca1-de23c5493a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:03.080784 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e8de0-896d-450b-bca1-de23c5493a38 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:03.081091 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e8de0-896d-450b-bca1-de23c5493a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:03.081341 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e8de0-896d-450b-bca1-de23c5493a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:03.082128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e8de0-896d-450b-bca1-de23c5493a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:03.083737 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e8de0-896d-450b-bca1-de23c5493a38 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:03.094835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1171/2290] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:03.098842 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a552d-12e9-4626-9746-46b3cba931e0 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:03.108694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a552d-12e9-4626-9746-46b3cba931e0 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:03.113530 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6ffef-3b23-43fe-b92c-ba3bd0708d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:03.114444 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6ffef-3b23-43fe-b92c-ba3bd0708d13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:03.114727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6ffef-3b23-43fe-b92c-ba3bd0708d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:03.115001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6ffef-3b23-43fe-b92c-ba3bd0708d13 admin admin] PATH_INFO: `/v3/projects/c9cc7dadf1c84452b3d9d659451d1543/users/a992d0b1593642feb6be30e3f02a8246/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:03.129638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6ffef-3b23-43fe-b92c-ba3bd0708d13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9cc7dadf1c84452b3d9d659451d1543, user_id=a992d0b1593642feb6be30e3f02a8246, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:03.132332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6ffef-3b23-43fe-b92c-ba3bd0708d13 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:03.140772 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:03.147173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1120/2291] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:03 2022] PUT /identity/v3/projects/c9cc7dadf1c84452b3d9d659451d1543/users/a992d0b1593642feb6be30e3f02a8246/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:03.151081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e8de0-896d-450b-bca1-de23c5493a38 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:03.161631 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e8de0-896d-450b-bca1-de23c5493a38 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:03.217031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497289e-dc58-472f-91d2-87a6a5d29287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:03.217612 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497289e-dc58-472f-91d2-87a6a5d29287 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:03.217897 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497289e-dc58-472f-91d2-87a6a5d29287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:03.218211 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497289e-dc58-472f-91d2-87a6a5d29287 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:03.218934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497289e-dc58-472f-91d2-87a6a5d29287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:03.220871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497289e-dc58-472f-91d2-87a6a5d29287 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:03.230357 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1172/2292] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:03.234397 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6ffef-3b23-43fe-b92c-ba3bd0708d13 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:03.244780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6ffef-3b23-43fe-b92c-ba3bd0708d13 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:03.249455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff0061-37b7-4070-ab6c-9f6bb805219b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:03.250373 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff0061-37b7-4070-ab6c-9f6bb805219b admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:03.250658 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff0061-37b7-4070-ab6c-9f6bb805219b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:03.250936 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff0061-37b7-4070-ab6c-9f6bb805219b admin admin] PATH_INFO: `/v3/projects/c9cc7dadf1c84452b3d9d659451d1543/users/a992d0b1593642feb6be30e3f02a8246/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:03.253499 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff0061-37b7-4070-ab6c-9f6bb805219b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9cc7dadf1c84452b3d9d659451d1543, user_id=a992d0b1593642feb6be30e3f02a8246, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:03.256184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff0061-37b7-4070-ab6c-9f6bb805219b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:03.266858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1121/2293] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:03 2022] PUT /identity/v3/projects/c9cc7dadf1c84452b3d9d659451d1543/users/a992d0b1593642feb6be30e3f02a8246/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:03.277505 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62057474-cf57-4c1c-b176-0aef274b170d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:03.277828 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62057474-cf57-4c1c-b176-0aef274b170d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:03.278142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62057474-cf57-4c1c-b176-0aef274b170d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:03.296860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:03.304224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-62057474-cf57-4c1c-b176-0aef274b170d None None] MFA Rules not processed for user `a992d0b1593642feb6be30e3f02a8246`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:03.305430 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-62057474-cf57-4c1c-b176-0aef274b170d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:03.376367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1173/2294] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:03 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:03.389529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff0061-37b7-4070-ab6c-9f6bb805219b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:03.391299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b2ff0061-37b7-4070-ab6c-9f6bb805219b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:03.447185 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff0061-37b7-4070-ab6c-9f6bb805219b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:03.450957 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f6651-4df6-4325-943a-5adc6eb70501 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:03.451686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f6651-4df6-4325-943a-5adc6eb70501 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:03.451978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f6651-4df6-4325-943a-5adc6eb70501 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:03.452254 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f6651-4df6-4325-943a-5adc6eb70501 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:03.461817 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f6651-4df6-4325-943a-5adc6eb70501 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:03.463493 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f6651-4df6-4325-943a-5adc6eb70501 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:03.473698 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1122/2295] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:03.776532 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62057474-cf57-4c1c-b176-0aef274b170d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:03.786567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62057474-cf57-4c1c-b176-0aef274b170d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:03.791600 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac0ed9-f191-40fe-a3e7-2f887df830d1 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a992d0b1593642feb6be30e3f02a8246', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9cc7dadf1c84452b3d9d659451d1543', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:03.792257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac0ed9-f191-40fe-a3e7-2f887df830d1 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:03.792547 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac0ed9-f191-40fe-a3e7-2f887df830d1 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:03.792827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac0ed9-f191-40fe-a3e7-2f887df830d1 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:03.793551 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac0ed9-f191-40fe-a3e7-2f887df830d1 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:48:03.793863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac0ed9-f191-40fe-a3e7-2f887df830d1 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:03.795506 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac0ed9-f191-40fe-a3e7-2f887df830d1 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:03.806810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1174/2296] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:48:03.810831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f6651-4df6-4325-943a-5adc6eb70501 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:03.820223 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f6651-4df6-4325-943a-5adc6eb70501 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:03.825287 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bde758-5a6a-4eab-84d5-1acc9ef2e3a5 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a992d0b1593642feb6be30e3f02a8246', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9cc7dadf1c84452b3d9d659451d1543', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:03.825936 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bde758-5a6a-4eab-84d5-1acc9ef2e3a5 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:03.826263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bde758-5a6a-4eab-84d5-1acc9ef2e3a5 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:03.826545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bde758-5a6a-4eab-84d5-1acc9ef2e3a5 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:03.827234 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bde758-5a6a-4eab-84d5-1acc9ef2e3a5 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:03.829121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bde758-5a6a-4eab-84d5-1acc9ef2e3a5 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:03.856343 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1123/2297] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:48:03 2022] POST /identity/v3/projects => generated 355 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:03.860420 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac0ed9-f191-40fe-a3e7-2f887df830d1 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:03.870635 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac0ed9-f191-40fe-a3e7-2f887df830d1 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:03.926102 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be048cd7-91d5-4baa-8766-d30762cd4d44 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a992d0b1593642feb6be30e3f02a8246', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9cc7dadf1c84452b3d9d659451d1543', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:03.926792 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be048cd7-91d5-4baa-8766-d30762cd4d44 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:03.927080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be048cd7-91d5-4baa-8766-d30762cd4d44 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:03.927355 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be048cd7-91d5-4baa-8766-d30762cd4d44 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:03.928072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be048cd7-91d5-4baa-8766-d30762cd4d44 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:48:03.928404 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be048cd7-91d5-4baa-8766-d30762cd4d44 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:03.930093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be048cd7-91d5-4baa-8766-d30762cd4d44 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:03.940947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1175/2298] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:48:03.976461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bde758-5a6a-4eab-84d5-1acc9ef2e3a5 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:03.986472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bde758-5a6a-4eab-84d5-1acc9ef2e3a5 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:03.991049 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7871cf-1f9a-4ecd-b27b-d80205d5f148 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a992d0b1593642feb6be30e3f02a8246', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9cc7dadf1c84452b3d9d659451d1543', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:03.991836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7871cf-1f9a-4ecd-b27b-d80205d5f148 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:03.992120 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7871cf-1f9a-4ecd-b27b-d80205d5f148 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:03.992395 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7871cf-1f9a-4ecd-b27b-d80205d5f148 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] PATH_INFO: `/v3/projects/12998877f60b41d5920a832e23e4ac5d` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:03.993584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7871cf-1f9a-4ecd-b27b-d80205d5f148 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] RBAC: Authorizing `identity:delete_project(project_id=12998877f60b41d5920a832e23e4ac5d)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:03.995715 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7871cf-1f9a-4ecd-b27b-d80205d5f148 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:04.048532 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dd7871cf-1f9a-4ecd-b27b-d80205d5f148 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12998877f60b41d5920a832e23e4ac5d', 'request_id': 'req-dd7871cf-1f9a-4ecd-b27b-d80205d5f148'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:04.053919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dd7871cf-1f9a-4ecd-b27b-d80205d5f148 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12998877f60b41d5920a832e23e4ac5d', 'request_id': 'req-dd7871cf-1f9a-4ecd-b27b-d80205d5f148'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:04.061747 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dd7871cf-1f9a-4ecd-b27b-d80205d5f148 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12998877f60b41d5920a832e23e4ac5d', 'request_id': 'req-dd7871cf-1f9a-4ecd-b27b-d80205d5f148'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:04.072625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dd7871cf-1f9a-4ecd-b27b-d80205d5f148 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] The token cache is being invalidate because project 12998877f60b41d5920a832e23e4ac5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:48:04.072901 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dd7871cf-1f9a-4ecd-b27b-d80205d5f148 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:04.074246 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dd7871cf-1f9a-4ecd-b27b-d80205d5f148 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:04.075814 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1124/2299] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:48:03 2022] DELETE /identity/v3/projects/12998877f60b41d5920a832e23e4ac5d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:04.254988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be048cd7-91d5-4baa-8766-d30762cd4d44 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:04.256751 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-be048cd7-91d5-4baa-8766-d30762cd4d44 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:04.321566 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be048cd7-91d5-4baa-8766-d30762cd4d44 tempest-VolumeQuotaClassesTest-1557525837 tempest-VolumeQuotaClassesTest-1557525837-project-admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:04.340823 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:04.376067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14c532-3b24-43be-b393-9b0977cbe41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:04.377173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14c532-3b24-43be-b393-9b0977cbe41f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:04.377454 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14c532-3b24-43be-b393-9b0977cbe41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:04.377724 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14c532-3b24-43be-b393-9b0977cbe41f admin admin] PATH_INFO: `/v3/users/193d69cecd9e4207b25a26a684c75acd` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:04.378925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14c532-3b24-43be-b393-9b0977cbe41f admin admin] RBAC: Authorizing `identity:delete_user(user_id=193d69cecd9e4207b25a26a684c75acd)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:04.380835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14c532-3b24-43be-b393-9b0977cbe41f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:04.425324 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9b14c532-3b24-43be-b393-9b0977cbe41f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '193d69cecd9e4207b25a26a684c75acd'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:04.425996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9b14c532-3b24-43be-b393-9b0977cbe41f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '193d69cecd9e4207b25a26a684c75acd'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:04.426606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9b14c532-3b24-43be-b393-9b0977cbe41f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '193d69cecd9e4207b25a26a684c75acd'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:04.439055 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9b14c532-3b24-43be-b393-9b0977cbe41f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '193d69cecd9e4207b25a26a684c75acd'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:04.445663 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9b14c532-3b24-43be-b393-9b0977cbe41f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '193d69cecd9e4207b25a26a684c75acd'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:04.458088 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1176/2300] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:48:04 2022] DELETE /identity/v3/users/193d69cecd9e4207b25a26a684c75acd => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:04.462549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d5ead-6e7c-40ea-a37b-d0d9e65aa3c7 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:04.463021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d5ead-6e7c-40ea-a37b-d0d9e65aa3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:04.463175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d5ead-6e7c-40ea-a37b-d0d9e65aa3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:04.480918 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-856d5ead-6e7c-40ea-a37b-d0d9e65aa3c7 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:04.482096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-856d5ead-6e7c-40ea-a37b-d0d9e65aa3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:04.528171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:04.546218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1125/2301] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:04.555940 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14c532-3b24-43be-b393-9b0977cbe41f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:04.557360 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9b14c532-3b24-43be-b393-9b0977cbe41f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:04.616856 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14c532-3b24-43be-b393-9b0977cbe41f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:04.621157 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed977b6-9dd5-4f2d-98fb-ec6a2c4a14fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:04.621800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed977b6-9dd5-4f2d-98fb-ec6a2c4a14fa service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:04.622110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed977b6-9dd5-4f2d-98fb-ec6a2c4a14fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:04.622390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed977b6-9dd5-4f2d-98fb-ec6a2c4a14fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:04.632000 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed977b6-9dd5-4f2d-98fb-ec6a2c4a14fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:04.633427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed977b6-9dd5-4f2d-98fb-ec6a2c4a14fa service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:04.643098 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1177/2302] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:04.901944 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d5ead-6e7c-40ea-a37b-d0d9e65aa3c7 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:04.903577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-856d5ead-6e7c-40ea-a37b-d0d9e65aa3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:04.916151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d5ead-6e7c-40ea-a37b-d0d9e65aa3c7 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:04.921625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79c294-e766-472f-a44d-98f089994ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:04.922351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79c294-e766-472f-a44d-98f089994ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:04.922655 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79c294-e766-472f-a44d-98f089994ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:04.922905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79c294-e766-472f-a44d-98f089994ce6 admin admin] PATH_INFO: `/v3/projects/7266a9f98bec48919e1b2417e9792ca6` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:04.924149 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79c294-e766-472f-a44d-98f089994ce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7266a9f98bec48919e1b2417e9792ca6)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:04.926057 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79c294-e766-472f-a44d-98f089994ce6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:04.978548 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3a79c294-e766-472f-a44d-98f089994ce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7266a9f98bec48919e1b2417e9792ca6', 'request_id': 'req-3a79c294-e766-472f-a44d-98f089994ce6'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:04.985526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3a79c294-e766-472f-a44d-98f089994ce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7266a9f98bec48919e1b2417e9792ca6', 'request_id': 'req-3a79c294-e766-472f-a44d-98f089994ce6'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:04.994376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3a79c294-e766-472f-a44d-98f089994ce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7266a9f98bec48919e1b2417e9792ca6', 'request_id': 'req-3a79c294-e766-472f-a44d-98f089994ce6'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:05.006844 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3a79c294-e766-472f-a44d-98f089994ce6 admin admin] The token cache is being invalidate because project 7266a9f98bec48919e1b2417e9792ca6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:48:05.007430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3a79c294-e766-472f-a44d-98f089994ce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:05.008308 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3a79c294-e766-472f-a44d-98f089994ce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:05.010453 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1126/2303] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:48:04 2022] DELETE /identity/v3/projects/7266a9f98bec48919e1b2417e9792ca6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:05.017670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed977b6-9dd5-4f2d-98fb-ec6a2c4a14fa service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:05.030776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed977b6-9dd5-4f2d-98fb-ec6a2c4a14fa service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:05.088426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:05.104114 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a20885-26db-4150-b5b5-a94a9423b8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:05.105654 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a20885-26db-4150-b5b5-a94a9423b8a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:05.105940 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a20885-26db-4150-b5b5-a94a9423b8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:05.106261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a20885-26db-4150-b5b5-a94a9423b8a6 admin admin] PATH_INFO: `/v3/users/a992d0b1593642feb6be30e3f02a8246` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:05.107485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a20885-26db-4150-b5b5-a94a9423b8a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a992d0b1593642feb6be30e3f02a8246)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:05.109428 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a20885-26db-4150-b5b5-a94a9423b8a6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:05.158333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-35a20885-26db-4150-b5b5-a94a9423b8a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a992d0b1593642feb6be30e3f02a8246'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:05.159154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-35a20885-26db-4150-b5b5-a94a9423b8a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a992d0b1593642feb6be30e3f02a8246'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:05.159874 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-35a20885-26db-4150-b5b5-a94a9423b8a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a992d0b1593642feb6be30e3f02a8246'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:05.172629 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-35a20885-26db-4150-b5b5-a94a9423b8a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a992d0b1593642feb6be30e3f02a8246'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:05.180142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-35a20885-26db-4150-b5b5-a94a9423b8a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a992d0b1593642feb6be30e3f02a8246'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:05.194604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1178/2304] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:48:05 2022] DELETE /identity/v3/users/a992d0b1593642feb6be30e3f02a8246 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:05.471292 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79c294-e766-472f-a44d-98f089994ce6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:05.472777 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3a79c294-e766-472f-a44d-98f089994ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:05.486313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79c294-e766-472f-a44d-98f089994ce6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:05.543998 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a8739-6a1b-48af-b4c4-7d271b2f80c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:05.544863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a8739-6a1b-48af-b4c4-7d271b2f80c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:05.545145 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a8739-6a1b-48af-b4c4-7d271b2f80c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:05.545419 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a8739-6a1b-48af-b4c4-7d271b2f80c1 admin admin] PATH_INFO: `/v3/projects/c9cc7dadf1c84452b3d9d659451d1543` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:05.546606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a8739-6a1b-48af-b4c4-7d271b2f80c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9cc7dadf1c84452b3d9d659451d1543)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:05.548534 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a8739-6a1b-48af-b4c4-7d271b2f80c1 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:05.594295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-260a8739-6a1b-48af-b4c4-7d271b2f80c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9cc7dadf1c84452b3d9d659451d1543', 'request_id': 'req-260a8739-6a1b-48af-b4c4-7d271b2f80c1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:05.599958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-260a8739-6a1b-48af-b4c4-7d271b2f80c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9cc7dadf1c84452b3d9d659451d1543', 'request_id': 'req-260a8739-6a1b-48af-b4c4-7d271b2f80c1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:05.610244 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-260a8739-6a1b-48af-b4c4-7d271b2f80c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9cc7dadf1c84452b3d9d659451d1543', 'request_id': 'req-260a8739-6a1b-48af-b4c4-7d271b2f80c1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:05.621932 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-260a8739-6a1b-48af-b4c4-7d271b2f80c1 admin admin] The token cache is being invalidate because project c9cc7dadf1c84452b3d9d659451d1543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:48:05.622255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-260a8739-6a1b-48af-b4c4-7d271b2f80c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:05.622877 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-260a8739-6a1b-48af-b4c4-7d271b2f80c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:05.624416 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1127/2305] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:48:05 2022] DELETE /identity/v3/projects/c9cc7dadf1c84452b3d9d659451d1543 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:05.629624 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1f067-da53-4aaa-a50b-da0ca9fe76f9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:05.630112 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1f067-da53-4aaa-a50b-da0ca9fe76f9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:05.630204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1f067-da53-4aaa-a50b-da0ca9fe76f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:05.648906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7bf1f067-da53-4aaa-a50b-da0ca9fe76f9 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:05.650208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7bf1f067-da53-4aaa-a50b-da0ca9fe76f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:05.711615 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1179/2306] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:05.721175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4e170-70ca-45b4-a88c-3abad4ba4cf4 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:05.721491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4e170-70ca-45b4-a88c-3abad4ba4cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:05.721795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4e170-70ca-45b4-a88c-3abad4ba4cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:05.728466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:05.744004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4dd4e170-70ca-45b4-a88c-3abad4ba4cf4 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:05.745354 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4dd4e170-70ca-45b4-a88c-3abad4ba4cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:05.754534 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1128/2307] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:05.758710 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1f067-da53-4aaa-a50b-da0ca9fe76f9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:05.768447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1f067-da53-4aaa-a50b-da0ca9fe76f9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:05.773111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441032b0-14af-4dcd-88f4-a35925f092a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:05.773771 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441032b0-14af-4dcd-88f4-a35925f092a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:05.774088 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441032b0-14af-4dcd-88f4-a35925f092a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:05.774366 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441032b0-14af-4dcd-88f4-a35925f092a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:05.775098 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441032b0-14af-4dcd-88f4-a35925f092a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:48:05.775408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441032b0-14af-4dcd-88f4-a35925f092a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:05.777557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441032b0-14af-4dcd-88f4-a35925f092a2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:05.789485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1180/2308] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:48:05.794595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4e170-70ca-45b4-a88c-3abad4ba4cf4 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:05.804162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4e170-70ca-45b4-a88c-3abad4ba4cf4 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:05.809022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bcac3-fbbc-45d0-971d-692a82b8910b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:05.809672 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bcac3-fbbc-45d0-971d-692a82b8910b admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:05.809975 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bcac3-fbbc-45d0-971d-692a82b8910b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:05.810264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bcac3-fbbc-45d0-971d-692a82b8910b admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:05.810957 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bcac3-fbbc-45d0-971d-692a82b8910b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:05.812843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bcac3-fbbc-45d0-971d-692a82b8910b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:05.838026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1129/2309] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:48:05 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:05.845738 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441032b0-14af-4dcd-88f4-a35925f092a2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:05.856553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441032b0-14af-4dcd-88f4-a35925f092a2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:05.903915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:05.914027 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553b433-7f46-4f15-9055-ae4a803b4c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:05.914692 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553b433-7f46-4f15-9055-ae4a803b4c70 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:05.914978 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553b433-7f46-4f15-9055-ae4a803b4c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:05.915254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553b433-7f46-4f15-9055-ae4a803b4c70 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:05.915944 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553b433-7f46-4f15-9055-ae4a803b4c70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:05.918541 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553b433-7f46-4f15-9055-ae4a803b4c70 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:05.937527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1181/2310] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:48:05 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:05.941583 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bcac3-fbbc-45d0-971d-692a82b8910b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:05.952306 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bcac3-fbbc-45d0-971d-692a82b8910b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:05.956847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb8de6-1dc2-40a9-bb18-bbbb25ec29c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:05.957491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb8de6-1dc2-40a9-bb18-bbbb25ec29c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:05.957768 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb8de6-1dc2-40a9-bb18-bbbb25ec29c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:05.958066 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb8de6-1dc2-40a9-bb18-bbbb25ec29c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:05.958784 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb8de6-1dc2-40a9-bb18-bbbb25ec29c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:05.960355 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb8de6-1dc2-40a9-bb18-bbbb25ec29c5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:05.970968 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1130/2311] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:05.974651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553b433-7f46-4f15-9055-ae4a803b4c70 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:05.985266 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553b433-7f46-4f15-9055-ae4a803b4c70 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:05.989952 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e8aa4-121e-487a-9deb-7ad45a94b6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:05.990631 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e8aa4-121e-487a-9deb-7ad45a94b6f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:05.990920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e8aa4-121e-487a-9deb-7ad45a94b6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:05.991208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e8aa4-121e-487a-9deb-7ad45a94b6f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:05.991923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e8aa4-121e-487a-9deb-7ad45a94b6f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:05.993526 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e8aa4-121e-487a-9deb-7ad45a94b6f5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:06.006608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1182/2312] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:06.009012 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb8de6-1dc2-40a9-bb18-bbbb25ec29c5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:06.019516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb8de6-1dc2-40a9-bb18-bbbb25ec29c5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:06.025537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4de96c-0302-46a0-8c38-c04cf69d6921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:06.027055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4de96c-0302-46a0-8c38-c04cf69d6921 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:06.027342 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4de96c-0302-46a0-8c38-c04cf69d6921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:06.027619 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4de96c-0302-46a0-8c38-c04cf69d6921 admin admin] PATH_INFO: `/v3/projects/cbea08ccd8d142a4a4dc209fe0726a3f/users/e41dc41f9832449591b9b7772c53975e/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:06.042060 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4de96c-0302-46a0-8c38-c04cf69d6921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbea08ccd8d142a4a4dc209fe0726a3f, user_id=e41dc41f9832449591b9b7772c53975e, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:06.044760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4de96c-0302-46a0-8c38-c04cf69d6921 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:06.057390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1131/2313] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:06 2022] PUT /identity/v3/projects/cbea08ccd8d142a4a4dc209fe0726a3f/users/e41dc41f9832449591b9b7772c53975e/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:06.067744 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9fb76-6a03-45f0-9918-b7ccb12cd10d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:06.068037 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9fb76-6a03-45f0-9918-b7ccb12cd10d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:06.068318 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9fb76-6a03-45f0-9918-b7ccb12cd10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:06.097034 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b7a9fb76-6a03-45f0-9918-b7ccb12cd10d None None] MFA Rules not processed for user `e41dc41f9832449591b9b7772c53975e`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:06.098299 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b7a9fb76-6a03-45f0-9918-b7ccb12cd10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:06.166655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1183/2314] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:06.171288 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4de96c-0302-46a0-8c38-c04cf69d6921 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:06.241553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4de96c-0302-46a0-8c38-c04cf69d6921 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:06.246883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5c17a-2c04-40a2-ae4d-4a265499acbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:06.247163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5c17a-2c04-40a2-ae4d-4a265499acbc admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:06.247452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5c17a-2c04-40a2-ae4d-4a265499acbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:06.247730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5c17a-2c04-40a2-ae4d-4a265499acbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:06.248434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5c17a-2c04-40a2-ae4d-4a265499acbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:06.250376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5c17a-2c04-40a2-ae4d-4a265499acbc admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:06.282398 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1132/2315] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:48:06 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:06.284764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9fb76-6a03-45f0-9918-b7ccb12cd10d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:06.296152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9fb76-6a03-45f0-9918-b7ccb12cd10d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:06.350927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f9e5c-a6f0-4353-af25-c1fc8f254b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:06.351592 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f9e5c-a6f0-4353-af25-c1fc8f254b5f admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:06.351880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f9e5c-a6f0-4353-af25-c1fc8f254b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:06.352167 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f9e5c-a6f0-4353-af25-c1fc8f254b5f admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:06.352862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f9e5c-a6f0-4353-af25-c1fc8f254b5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:06.354860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f9e5c-a6f0-4353-af25-c1fc8f254b5f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:06.375072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1184/2316] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:48:06 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:06.379424 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5c17a-2c04-40a2-ae4d-4a265499acbc admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:06.390758 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5c17a-2c04-40a2-ae4d-4a265499acbc admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:06.395533 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8f63d-6611-4304-adb0-265c8eccc9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:06.396195 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8f63d-6611-4304-adb0-265c8eccc9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:06.396482 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8f63d-6611-4304-adb0-265c8eccc9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:06.396759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8f63d-6611-4304-adb0-265c8eccc9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:06.397532 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8f63d-6611-4304-adb0-265c8eccc9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:06.399299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8f63d-6611-4304-adb0-265c8eccc9f3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:06.413180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1133/2317] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:06.414895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f9e5c-a6f0-4353-af25-c1fc8f254b5f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:06.424285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f9e5c-a6f0-4353-af25-c1fc8f254b5f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:06.429096 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1e7fe-c461-4fbc-9828-3f6f88bf611e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:06.430007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1e7fe-c461-4fbc-9828-3f6f88bf611e admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:06.430301 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1e7fe-c461-4fbc-9828-3f6f88bf611e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:06.430582 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1e7fe-c461-4fbc-9828-3f6f88bf611e admin admin] PATH_INFO: `/v3/projects/533f77e2c78f416c8723b0acf4860fc8/users/d8cb430509624986aa9368a0eb6a6186/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:06.446183 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1e7fe-c461-4fbc-9828-3f6f88bf611e admin admin] RBAC: Authorizing `identity:create_grant(project_id=533f77e2c78f416c8723b0acf4860fc8, user_id=d8cb430509624986aa9368a0eb6a6186, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:06.449454 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1e7fe-c461-4fbc-9828-3f6f88bf611e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:06.461916 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1185/2318] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:06 2022] PUT /identity/v3/projects/533f77e2c78f416c8723b0acf4860fc8/users/d8cb430509624986aa9368a0eb6a6186/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:06.466071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8f63d-6611-4304-adb0-265c8eccc9f3 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:06.476040 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8f63d-6611-4304-adb0-265c8eccc9f3 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:06.531360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace79a98-5a3a-4941-b47f-f6fea9b9d3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:06.532324 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace79a98-5a3a-4941-b47f-f6fea9b9d3a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:06.532612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace79a98-5a3a-4941-b47f-f6fea9b9d3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:06.532889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace79a98-5a3a-4941-b47f-f6fea9b9d3a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:06.533612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace79a98-5a3a-4941-b47f-f6fea9b9d3a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:06.535311 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace79a98-5a3a-4941-b47f-f6fea9b9d3a9 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:06.544871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1134/2319] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:06.549082 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1e7fe-c461-4fbc-9828-3f6f88bf611e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:06.559753 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1e7fe-c461-4fbc-9828-3f6f88bf611e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:06.564353 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448abe19-0615-497e-b2fe-13411192e7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:06.565227 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448abe19-0615-497e-b2fe-13411192e7f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:06.565514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448abe19-0615-497e-b2fe-13411192e7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:06.565793 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448abe19-0615-497e-b2fe-13411192e7f6 admin admin] PATH_INFO: `/v3/projects/533f77e2c78f416c8723b0acf4860fc8/users/d8cb430509624986aa9368a0eb6a6186/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:06.567811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448abe19-0615-497e-b2fe-13411192e7f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=533f77e2c78f416c8723b0acf4860fc8, user_id=d8cb430509624986aa9368a0eb6a6186, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:06.570491 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448abe19-0615-497e-b2fe-13411192e7f6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:06.580991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1186/2320] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:06 2022] PUT /identity/v3/projects/533f77e2c78f416c8723b0acf4860fc8/users/d8cb430509624986aa9368a0eb6a6186/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:06.591030 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b96d0a-c3d0-4cbe-a98a-e003f6f8121b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:06.591359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b96d0a-c3d0-4cbe-a98a-e003f6f8121b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:06.591643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b96d0a-c3d0-4cbe-a98a-e003f6f8121b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:06.609948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:06.619892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e9b96d0a-c3d0-4cbe-a98a-e003f6f8121b None None] MFA Rules not processed for user `d8cb430509624986aa9368a0eb6a6186`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:06.621672 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e9b96d0a-c3d0-4cbe-a98a-e003f6f8121b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:06.688911 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1135/2321] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:06 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:06.699942 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448abe19-0615-497e-b2fe-13411192e7f6 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:06.701714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-448abe19-0615-497e-b2fe-13411192e7f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:06.715986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:06.759642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448abe19-0615-497e-b2fe-13411192e7f6 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:06.763359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56687261-19ac-4885-a050-1f4e3ebe0260 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:06.763991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56687261-19ac-4885-a050-1f4e3ebe0260 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:06.764277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56687261-19ac-4885-a050-1f4e3ebe0260 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:06.764554 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56687261-19ac-4885-a050-1f4e3ebe0260 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:06.819927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56687261-19ac-4885-a050-1f4e3ebe0260 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:06.822727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56687261-19ac-4885-a050-1f4e3ebe0260 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:06.830509 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1187/2322] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:11.586159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b96d0a-c3d0-4cbe-a98a-e003f6f8121b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:11.653200 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b96d0a-c3d0-4cbe-a98a-e003f6f8121b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:11.657759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997d506-2040-446b-8ab5-7481eb952a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:11.658671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997d506-2040-446b-8ab5-7481eb952a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:11.658951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997d506-2040-446b-8ab5-7481eb952a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:11.659227 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997d506-2040-446b-8ab5-7481eb952a49 admin admin] PATH_INFO: `/v3/users/e41dc41f9832449591b9b7772c53975e` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:11.660814 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5997d506-2040-446b-8ab5-7481eb952a49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e41dc41f9832449591b9b7772c53975e)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:11.662776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5997d506-2040-446b-8ab5-7481eb952a49 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:11.723706 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-5997d506-2040-446b-8ab5-7481eb952a49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e41dc41f9832449591b9b7772c53975e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:11.737688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-5997d506-2040-446b-8ab5-7481eb952a49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e41dc41f9832449591b9b7772c53975e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:11.746168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-5997d506-2040-446b-8ab5-7481eb952a49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e41dc41f9832449591b9b7772c53975e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:11.746906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-5997d506-2040-446b-8ab5-7481eb952a49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e41dc41f9832449591b9b7772c53975e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:11.757028 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-5997d506-2040-446b-8ab5-7481eb952a49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e41dc41f9832449591b9b7772c53975e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:11.764360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1136/2323] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:48:11 2022] DELETE /identity/v3/users/e41dc41f9832449591b9b7772c53975e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:11.768818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e6f2d-0270-456b-9add-bd92d2d8d09b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:11.769295 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e6f2d-0270-456b-9add-bd92d2d8d09b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:11.769435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e6f2d-0270-456b-9add-bd92d2d8d09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:11.787956 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9f0e6f2d-0270-456b-9add-bd92d2d8d09b None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:11.789415 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9f0e6f2d-0270-456b-9add-bd92d2d8d09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:11.860900 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1188/2324] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:11.873044 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997d506-2040-446b-8ab5-7481eb952a49 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:11.879071 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5997d506-2040-446b-8ab5-7481eb952a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:11.895919 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:11.963025 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997d506-2040-446b-8ab5-7481eb952a49 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:11.967437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897e5a5-7711-419a-9572-62a4c5c5aeb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:11.968165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897e5a5-7711-419a-9572-62a4c5c5aeb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:11.968516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897e5a5-7711-419a-9572-62a4c5c5aeb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:11.968859 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897e5a5-7711-419a-9572-62a4c5c5aeb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:11.978315 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a897e5a5-7711-419a-9572-62a4c5c5aeb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:11.979806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a897e5a5-7711-419a-9572-62a4c5c5aeb4 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:11.989451 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1137/2325] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:12.252854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e6f2d-0270-456b-9add-bd92d2d8d09b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:12.254464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9f0e6f2d-0270-456b-9add-bd92d2d8d09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:12.268143 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e6f2d-0270-456b-9add-bd92d2d8d09b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:12.274035 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca58298-2db6-47a1-8f0d-51152bbfdc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:12.275254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca58298-2db6-47a1-8f0d-51152bbfdc1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:12.275254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca58298-2db6-47a1-8f0d-51152bbfdc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:12.275428 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca58298-2db6-47a1-8f0d-51152bbfdc1d admin admin] PATH_INFO: `/v3/projects/cbea08ccd8d142a4a4dc209fe0726a3f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:12.277979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca58298-2db6-47a1-8f0d-51152bbfdc1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbea08ccd8d142a4a4dc209fe0726a3f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:12.279953 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca58298-2db6-47a1-8f0d-51152bbfdc1d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:12.327767 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-aca58298-2db6-47a1-8f0d-51152bbfdc1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbea08ccd8d142a4a4dc209fe0726a3f', 'request_id': 'req-aca58298-2db6-47a1-8f0d-51152bbfdc1d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:12.334972 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-aca58298-2db6-47a1-8f0d-51152bbfdc1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbea08ccd8d142a4a4dc209fe0726a3f', 'request_id': 'req-aca58298-2db6-47a1-8f0d-51152bbfdc1d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:12.341251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-aca58298-2db6-47a1-8f0d-51152bbfdc1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbea08ccd8d142a4a4dc209fe0726a3f', 'request_id': 'req-aca58298-2db6-47a1-8f0d-51152bbfdc1d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:12.343299 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:12.354156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-aca58298-2db6-47a1-8f0d-51152bbfdc1d admin admin] The token cache is being invalidate because project cbea08ccd8d142a4a4dc209fe0726a3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:48:12.354447 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-aca58298-2db6-47a1-8f0d-51152bbfdc1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:12.355076 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-aca58298-2db6-47a1-8f0d-51152bbfdc1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:12.356613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1189/2326] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:48:12 2022] DELETE /identity/v3/projects/cbea08ccd8d142a4a4dc209fe0726a3f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:12.360612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897e5a5-7711-419a-9572-62a4c5c5aeb4 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:12.370668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897e5a5-7711-419a-9572-62a4c5c5aeb4 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:12.424191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865615c-eef5-4fde-bb28-2e71585709e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:12.425042 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865615c-eef5-4fde-bb28-2e71585709e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:12.425328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865615c-eef5-4fde-bb28-2e71585709e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:12.425599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865615c-eef5-4fde-bb28-2e71585709e5 admin admin] PATH_INFO: `/v3/users/d8cb430509624986aa9368a0eb6a6186` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:12.426812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865615c-eef5-4fde-bb28-2e71585709e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8cb430509624986aa9368a0eb6a6186)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:12.428704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865615c-eef5-4fde-bb28-2e71585709e5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:12.472553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e865615c-eef5-4fde-bb28-2e71585709e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8cb430509624986aa9368a0eb6a6186'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:12.484162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e865615c-eef5-4fde-bb28-2e71585709e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8cb430509624986aa9368a0eb6a6186'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:12.490986 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e865615c-eef5-4fde-bb28-2e71585709e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8cb430509624986aa9368a0eb6a6186'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:12.491637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e865615c-eef5-4fde-bb28-2e71585709e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8cb430509624986aa9368a0eb6a6186'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:12.499427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e865615c-eef5-4fde-bb28-2e71585709e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8cb430509624986aa9368a0eb6a6186'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:12.505606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1138/2327] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:48:12 2022] DELETE /identity/v3/users/d8cb430509624986aa9368a0eb6a6186 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:12.778852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca58298-2db6-47a1-8f0d-51152bbfdc1d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:12.780304 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-aca58298-2db6-47a1-8f0d-51152bbfdc1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:12.795351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca58298-2db6-47a1-8f0d-51152bbfdc1d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:12.854763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8bf1b-9416-4fdb-8a4b-4613f0c54a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:12.855665 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8bf1b-9416-4fdb-8a4b-4613f0c54a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:12.856071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8bf1b-9416-4fdb-8a4b-4613f0c54a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:12.856359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8bf1b-9416-4fdb-8a4b-4613f0c54a85 admin admin] PATH_INFO: `/v3/projects/533f77e2c78f416c8723b0acf4860fc8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:12.857561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8bf1b-9416-4fdb-8a4b-4613f0c54a85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=533f77e2c78f416c8723b0acf4860fc8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:12.859517 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8bf1b-9416-4fdb-8a4b-4613f0c54a85 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:12.909681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-57a8bf1b-9416-4fdb-8a4b-4613f0c54a85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '533f77e2c78f416c8723b0acf4860fc8', 'request_id': 'req-57a8bf1b-9416-4fdb-8a4b-4613f0c54a85'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:12.917709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-57a8bf1b-9416-4fdb-8a4b-4613f0c54a85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '533f77e2c78f416c8723b0acf4860fc8', 'request_id': 'req-57a8bf1b-9416-4fdb-8a4b-4613f0c54a85'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:12.924786 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-57a8bf1b-9416-4fdb-8a4b-4613f0c54a85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '533f77e2c78f416c8723b0acf4860fc8', 'request_id': 'req-57a8bf1b-9416-4fdb-8a4b-4613f0c54a85'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:12.935833 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-57a8bf1b-9416-4fdb-8a4b-4613f0c54a85 admin admin] The token cache is being invalidate because project 533f77e2c78f416c8723b0acf4860fc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:48:12.936121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-57a8bf1b-9416-4fdb-8a4b-4613f0c54a85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:12.936786 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-57a8bf1b-9416-4fdb-8a4b-4613f0c54a85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:12.938364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1190/2328] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:48:12 2022] DELETE /identity/v3/projects/533f77e2c78f416c8723b0acf4860fc8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:12.943388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895449b3-0766-42b3-b6ea-803fd79c02cf None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:12.943883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895449b3-0766-42b3-b6ea-803fd79c02cf None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:12.944023 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895449b3-0766-42b3-b6ea-803fd79c02cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:12.961380 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-895449b3-0766-42b3-b6ea-803fd79c02cf None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:12.962647 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-895449b3-0766-42b3-b6ea-803fd79c02cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:12.993447 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:13.025473 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1139/2329] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:48:13.032888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e5370-5920-4a55-ab60-e93eae493cf7 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:13.033177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e5370-5920-4a55-ab60-e93eae493cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:13.033454 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e5370-5920-4a55-ab60-e93eae493cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:13.051949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b31e5370-5920-4a55-ab60-e93eae493cf7 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:13.053151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b31e5370-5920-4a55-ab60-e93eae493cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:13.063275 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1191/2330] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:13.067158 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895449b3-0766-42b3-b6ea-803fd79c02cf None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:13.077388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895449b3-0766-42b3-b6ea-803fd79c02cf None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:13.082130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cb305-a504-4a68-ad48-0d9f94315290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:13.082791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cb305-a504-4a68-ad48-0d9f94315290 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:13.083074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cb305-a504-4a68-ad48-0d9f94315290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:13.083352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cb305-a504-4a68-ad48-0d9f94315290 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:13.084085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2cb305-a504-4a68-ad48-0d9f94315290 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:48:13.084397 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2cb305-a504-4a68-ad48-0d9f94315290 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:13.086019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2cb305-a504-4a68-ad48-0d9f94315290 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:13.098300 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1140/2331] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:48:13.102622 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e5370-5920-4a55-ab60-e93eae493cf7 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:13.112836 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e5370-5920-4a55-ab60-e93eae493cf7 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:13.117417 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114a7f9-9d57-44d7-b54c-924e9eb58aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:13.118102 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114a7f9-9d57-44d7-b54c-924e9eb58aac admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:13.118384 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114a7f9-9d57-44d7-b54c-924e9eb58aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:13.118656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114a7f9-9d57-44d7-b54c-924e9eb58aac admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:13.119339 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114a7f9-9d57-44d7-b54c-924e9eb58aac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:13.121287 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114a7f9-9d57-44d7-b54c-924e9eb58aac admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:13.148092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1192/2332] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:48:13 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:13.152817 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cb305-a504-4a68-ad48-0d9f94315290 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:13.162902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cb305-a504-4a68-ad48-0d9f94315290 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:13.220659 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691268a-b9c6-45ca-9de4-db7fd255f062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:13.221355 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691268a-b9c6-45ca-9de4-db7fd255f062 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:13.221759 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691268a-b9c6-45ca-9de4-db7fd255f062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:13.221892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691268a-b9c6-45ca-9de4-db7fd255f062 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:13.222643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691268a-b9c6-45ca-9de4-db7fd255f062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:13.224578 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691268a-b9c6-45ca-9de4-db7fd255f062 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:13.244776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1141/2333] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:48:13 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:48:13.248831 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114a7f9-9d57-44d7-b54c-924e9eb58aac admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:13.259461 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114a7f9-9d57-44d7-b54c-924e9eb58aac admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:13.264013 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bafcc-bce8-410e-95ed-45d729c2198b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:13.264661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bafcc-bce8-410e-95ed-45d729c2198b admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:13.264941 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bafcc-bce8-410e-95ed-45d729c2198b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:13.265213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bafcc-bce8-410e-95ed-45d729c2198b admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:13.265928 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798bafcc-bce8-410e-95ed-45d729c2198b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:13.267550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798bafcc-bce8-410e-95ed-45d729c2198b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:13.278419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1193/2334] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:13.282018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691268a-b9c6-45ca-9de4-db7fd255f062 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:13.293681 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691268a-b9c6-45ca-9de4-db7fd255f062 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:13.298369 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a903c7d-114d-4ad6-bb32-154920193b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:13.299021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a903c7d-114d-4ad6-bb32-154920193b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:13.299302 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a903c7d-114d-4ad6-bb32-154920193b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:13.299578 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a903c7d-114d-4ad6-bb32-154920193b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:13.300326 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a903c7d-114d-4ad6-bb32-154920193b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:13.301906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a903c7d-114d-4ad6-bb32-154920193b0f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:13.311766 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1142/2335] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:13.315655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bafcc-bce8-410e-95ed-45d729c2198b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:13.325602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bafcc-bce8-410e-95ed-45d729c2198b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:13.330257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8c33a-0f4f-47c8-abb6-38c481924624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:13.331124 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8c33a-0f4f-47c8-abb6-38c481924624 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:13.331406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8c33a-0f4f-47c8-abb6-38c481924624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:13.331681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8c33a-0f4f-47c8-abb6-38c481924624 admin admin] PATH_INFO: `/v3/projects/31b63a8437804ab79bd2d3b9d858ed8b/users/c069545cf93d4fdcb89accf145441303/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:13.346288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8c33a-0f4f-47c8-abb6-38c481924624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b63a8437804ab79bd2d3b9d858ed8b, user_id=c069545cf93d4fdcb89accf145441303, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:13.348975 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8c33a-0f4f-47c8-abb6-38c481924624 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:13.362745 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1194/2336] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:13 2022] PUT /identity/v3/projects/31b63a8437804ab79bd2d3b9d858ed8b/users/c069545cf93d4fdcb89accf145441303/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:13.372965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31d54d-7050-431f-9ef9-078a9c6087fe None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:13.373253 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31d54d-7050-431f-9ef9-078a9c6087fe None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:13.373560 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31d54d-7050-431f-9ef9-078a9c6087fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:13.399624 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4e31d54d-7050-431f-9ef9-078a9c6087fe None None] MFA Rules not processed for user `c069545cf93d4fdcb89accf145441303`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:13.400817 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4e31d54d-7050-431f-9ef9-078a9c6087fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:13.464663 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1143/2337] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:13.469064 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8c33a-0f4f-47c8-abb6-38c481924624 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:13.487809 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:13.533330 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8c33a-0f4f-47c8-abb6-38c481924624 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:13.537987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf61076-592a-4a5f-ac7f-b5275f320d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:13.538647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf61076-592a-4a5f-ac7f-b5275f320d2c admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:13.538935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf61076-592a-4a5f-ac7f-b5275f320d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:13.539214 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf61076-592a-4a5f-ac7f-b5275f320d2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:13.539897 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf61076-592a-4a5f-ac7f-b5275f320d2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:13.541845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf61076-592a-4a5f-ac7f-b5275f320d2c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:13.566278 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1195/2338] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:48:13 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:13.569934 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31d54d-7050-431f-9ef9-078a9c6087fe None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:13.580803 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31d54d-7050-431f-9ef9-078a9c6087fe None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:13.609400 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:13.637634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cf358-ec8f-407c-b036-27379945fb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:13.638318 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cf358-ec8f-407c-b036-27379945fb9f admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:13.638597 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cf358-ec8f-407c-b036-27379945fb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:13.638867 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cf358-ec8f-407c-b036-27379945fb9f admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:13.639548 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161cf358-ec8f-407c-b036-27379945fb9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:13.641439 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161cf358-ec8f-407c-b036-27379945fb9f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:13.660584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1144/2339] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:48:13 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:13.664574 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf61076-592a-4a5f-ac7f-b5275f320d2c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:13.675630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf61076-592a-4a5f-ac7f-b5275f320d2c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:13.680282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c0213-3398-46fd-8e24-bbb33cc922b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:13.680939 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c0213-3398-46fd-8e24-bbb33cc922b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:13.681224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c0213-3398-46fd-8e24-bbb33cc922b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:13.681772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c0213-3398-46fd-8e24-bbb33cc922b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:13.683196 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c0213-3398-46fd-8e24-bbb33cc922b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:13.684890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c0213-3398-46fd-8e24-bbb33cc922b7 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:13.696837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1196/2340] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:13.700908 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cf358-ec8f-407c-b036-27379945fb9f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:13.711094 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cf358-ec8f-407c-b036-27379945fb9f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:13.715756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737aed8-3162-4bd4-aa9a-9b04826d12c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:13.716402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737aed8-3162-4bd4-aa9a-9b04826d12c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:13.716681 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737aed8-3162-4bd4-aa9a-9b04826d12c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:13.716951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737aed8-3162-4bd4-aa9a-9b04826d12c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:13.717668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737aed8-3162-4bd4-aa9a-9b04826d12c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:13.719285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737aed8-3162-4bd4-aa9a-9b04826d12c4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:13.729007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1145/2341] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:13.733700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c0213-3398-46fd-8e24-bbb33cc922b7 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:13.743729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c0213-3398-46fd-8e24-bbb33cc922b7 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:13.748982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27ec6e-a645-4f58-8983-9369edf67d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:13.749859 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27ec6e-a645-4f58-8983-9369edf67d41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:13.750180 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27ec6e-a645-4f58-8983-9369edf67d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:13.750461 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27ec6e-a645-4f58-8983-9369edf67d41 admin admin] PATH_INFO: `/v3/projects/a76a451c5ede4e4a80b349ef10ec075e/users/b6206c760eef4056a561212b38ab097d/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:13.765177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27ec6e-a645-4f58-8983-9369edf67d41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a76a451c5ede4e4a80b349ef10ec075e, user_id=b6206c760eef4056a561212b38ab097d, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:13.767879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27ec6e-a645-4f58-8983-9369edf67d41 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:13.780670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1197/2342] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:13 2022] PUT /identity/v3/projects/a76a451c5ede4e4a80b349ef10ec075e/users/b6206c760eef4056a561212b38ab097d/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:13.791562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00508ec8-f9cd-4559-9d6d-4fbbad8b7458 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:13.792396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00508ec8-f9cd-4559-9d6d-4fbbad8b7458 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:13.792686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00508ec8-f9cd-4559-9d6d-4fbbad8b7458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:13.819753 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-00508ec8-f9cd-4559-9d6d-4fbbad8b7458 None None] MFA Rules not processed for user `b6206c760eef4056a561212b38ab097d`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:13.820973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-00508ec8-f9cd-4559-9d6d-4fbbad8b7458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:13.886164 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1146/2343] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:13.890503 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27ec6e-a645-4f58-8983-9369edf67d41 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:13.951426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27ec6e-a645-4f58-8983-9369edf67d41 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:13.956048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0910ff7-09e8-492d-8d67-6518f17424bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:13.956694 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0910ff7-09e8-492d-8d67-6518f17424bc admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:13.956979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0910ff7-09e8-492d-8d67-6518f17424bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:13.957255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0910ff7-09e8-492d-8d67-6518f17424bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:13.957938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0910ff7-09e8-492d-8d67-6518f17424bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:13.959872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0910ff7-09e8-492d-8d67-6518f17424bc admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:13.984916 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1198/2344] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:48:13 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:13.988993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00508ec8-f9cd-4559-9d6d-4fbbad8b7458 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:13.999656 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00508ec8-f9cd-4559-9d6d-4fbbad8b7458 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:14.054673 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccf4bc-9acf-4738-96e3-30865894aa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:14.055333 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccf4bc-9acf-4738-96e3-30865894aa24 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:14.055610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccf4bc-9acf-4738-96e3-30865894aa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:14.055881 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccf4bc-9acf-4738-96e3-30865894aa24 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:14.056568 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bccf4bc-9acf-4738-96e3-30865894aa24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:14.058552 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bccf4bc-9acf-4738-96e3-30865894aa24 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:14.082475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1147/2345] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:48:13 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:14.083671 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0910ff7-09e8-492d-8d67-6518f17424bc admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:14.095131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0910ff7-09e8-492d-8d67-6518f17424bc admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:14.099890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8036195-c6cc-4ede-9e55-eb72716d4cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:14.100569 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8036195-c6cc-4ede-9e55-eb72716d4cff admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:14.100841 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8036195-c6cc-4ede-9e55-eb72716d4cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:14.101122 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8036195-c6cc-4ede-9e55-eb72716d4cff admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:14.101854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8036195-c6cc-4ede-9e55-eb72716d4cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:14.103494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8036195-c6cc-4ede-9e55-eb72716d4cff admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:14.114847 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1199/2346] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:14.118874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccf4bc-9acf-4738-96e3-30865894aa24 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:14.129148 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccf4bc-9acf-4738-96e3-30865894aa24 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:14.133785 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3b223-7306-4645-843e-18a0cd515c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:14.134692 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3b223-7306-4645-843e-18a0cd515c46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:14.134972 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3b223-7306-4645-843e-18a0cd515c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:14.135243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3b223-7306-4645-843e-18a0cd515c46 admin admin] PATH_INFO: `/v3/projects/a356ccce83e3442bad4ba30c50cf28e7/users/72dc864700234a6d8af419366dc83f49/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:14.151894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3b223-7306-4645-843e-18a0cd515c46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a356ccce83e3442bad4ba30c50cf28e7, user_id=72dc864700234a6d8af419366dc83f49, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:14.154819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3b223-7306-4645-843e-18a0cd515c46 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:14.169715 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1148/2347] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:14 2022] PUT /identity/v3/projects/a356ccce83e3442bad4ba30c50cf28e7/users/72dc864700234a6d8af419366dc83f49/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:14.173643 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8036195-c6cc-4ede-9e55-eb72716d4cff admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:14.184164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8036195-c6cc-4ede-9e55-eb72716d4cff admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:14.225708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:14.243274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233c4f0-6d90-4470-8c8b-6aaaa314e870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:14.244197 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233c4f0-6d90-4470-8c8b-6aaaa314e870 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:14.244714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233c4f0-6d90-4470-8c8b-6aaaa314e870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:14.244904 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233c4f0-6d90-4470-8c8b-6aaaa314e870 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:14.245935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233c4f0-6d90-4470-8c8b-6aaaa314e870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:14.247816 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233c4f0-6d90-4470-8c8b-6aaaa314e870 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:14.274276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1200/2348] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:14.279378 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3b223-7306-4645-843e-18a0cd515c46 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:14.288495 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:14.293427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3b223-7306-4645-843e-18a0cd515c46 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:14.300133 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b3679-576e-448f-af2a-ee9dc71c893e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:14.301269 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b3679-576e-448f-af2a-ee9dc71c893e admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:14.301984 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b3679-576e-448f-af2a-ee9dc71c893e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:14.302711 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b3679-576e-448f-af2a-ee9dc71c893e admin admin] PATH_INFO: `/v3/projects/a356ccce83e3442bad4ba30c50cf28e7/users/72dc864700234a6d8af419366dc83f49/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:14.307299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b3679-576e-448f-af2a-ee9dc71c893e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a356ccce83e3442bad4ba30c50cf28e7, user_id=72dc864700234a6d8af419366dc83f49, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:14.313034 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b3679-576e-448f-af2a-ee9dc71c893e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:14.330609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1149/2349] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:14 2022] PUT /identity/v3/projects/a356ccce83e3442bad4ba30c50cf28e7/users/72dc864700234a6d8af419366dc83f49/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:14.341585 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bcc9c-c8a4-4bcf-a1ee-c207605c39bd None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:14.342067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bcc9c-c8a4-4bcf-a1ee-c207605c39bd None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:14.342526 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bcc9c-c8a4-4bcf-a1ee-c207605c39bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:14.380421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-db9bcc9c-c8a4-4bcf-a1ee-c207605c39bd None None] MFA Rules not processed for user `72dc864700234a6d8af419366dc83f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:14.381882 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-db9bcc9c-c8a4-4bcf-a1ee-c207605c39bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:14.475007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1201/2350] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:14 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:14.486354 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b3679-576e-448f-af2a-ee9dc71c893e admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:14.488103 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-401b3679-576e-448f-af2a-ee9dc71c893e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:14.549271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b3679-576e-448f-af2a-ee9dc71c893e admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:14.552939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126b1c9-985f-4163-89cf-07cd1a3d7fcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:14.553581 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126b1c9-985f-4163-89cf-07cd1a3d7fcd service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:14.553867 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126b1c9-985f-4163-89cf-07cd1a3d7fcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:14.554175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126b1c9-985f-4163-89cf-07cd1a3d7fcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:14.616689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126b1c9-985f-4163-89cf-07cd1a3d7fcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:14.618170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126b1c9-985f-4163-89cf-07cd1a3d7fcd service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:14.628075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1150/2351] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:15.319697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514e4b3-1bf6-418e-bdff-e12089b09070 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:15.320586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514e4b3-1bf6-418e-bdff-e12089b09070 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:15.321050 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514e4b3-1bf6-418e-bdff-e12089b09070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:15.343531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7514e4b3-1bf6-418e-bdff-e12089b09070 None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:15.345179 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7514e4b3-1bf6-418e-bdff-e12089b09070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:15.436963 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1202/2352] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jul 12 13:48:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:15.505324 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126b1c9-985f-4163-89cf-07cd1a3d7fcd service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:15.506830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4126b1c9-985f-4163-89cf-07cd1a3d7fcd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:15.668211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126b1c9-985f-4163-89cf-07cd1a3d7fcd service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:15.675378 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e729e-1b12-4942-b4d1-2f5c5fb08c3b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:15.676235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e729e-1b12-4942-b4d1-2f5c5fb08c3b service swift] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:15.676626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e729e-1b12-4942-b4d1-2f5c5fb08c3b service swift] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:15.677107 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e729e-1b12-4942-b4d1-2f5c5fb08c3b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:15.695981 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e729e-1b12-4942-b4d1-2f5c5fb08c3b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:15.700731 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e729e-1b12-4942-b4d1-2f5c5fb08c3b service swift] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:15.721610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1151/2353] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:48:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:48:16.222175 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514e4b3-1bf6-418e-bdff-e12089b09070 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:16.223604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7514e4b3-1bf6-418e-bdff-e12089b09070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:16.236631 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514e4b3-1bf6-418e-bdff-e12089b09070 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:16.240198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce1f83-ef9a-4715-900d-1c5365a4304e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:16.240839 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce1f83-ef9a-4715-900d-1c5365a4304e service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:16.241120 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce1f83-ef9a-4715-900d-1c5365a4304e service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:16.241391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce1f83-ef9a-4715-900d-1c5365a4304e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:16.303124 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ce1f83-ef9a-4715-900d-1c5365a4304e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:16.304607 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ce1f83-ef9a-4715-900d-1c5365a4304e service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:16.315359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1203/2354] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:17.194885 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e729e-1b12-4942-b4d1-2f5c5fb08c3b service swift] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:17.205951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e729e-1b12-4942-b4d1-2f5c5fb08c3b service swift] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:17.210121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc497db-0427-45d3-8cb5-80a5168f2963 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:17.210880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc497db-0427-45d3-8cb5-80a5168f2963 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:17.211226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc497db-0427-45d3-8cb5-80a5168f2963 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:17.211587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc497db-0427-45d3-8cb5-80a5168f2963 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:17.222961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc497db-0427-45d3-8cb5-80a5168f2963 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:17.224412 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc497db-0427-45d3-8cb5-80a5168f2963 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:17.235664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1152/2355] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:48:18.253365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce1f83-ef9a-4715-900d-1c5365a4304e service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:18.255433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-22ce1f83-ef9a-4715-900d-1c5365a4304e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:18.267894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:18.322617 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce1f83-ef9a-4715-900d-1c5365a4304e service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:18.328262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2bafa-30e6-466d-a7aa-087bb288844a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:18.328978 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2bafa-30e6-466d-a7aa-087bb288844a service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:18.329328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2bafa-30e6-466d-a7aa-087bb288844a service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:18.329703 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2bafa-30e6-466d-a7aa-087bb288844a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:18.331702 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b5f2bafa-30e6-466d-a7aa-087bb288844a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:18.425306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2bafa-30e6-466d-a7aa-087bb288844a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:18.426805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2bafa-30e6-466d-a7aa-087bb288844a service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:18.440173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1204/2356] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:20.423063 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc497db-0427-45d3-8cb5-80a5168f2963 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:20.424716 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fcc497db-0427-45d3-8cb5-80a5168f2963 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:20.445479 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc497db-0427-45d3-8cb5-80a5168f2963 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:20.453017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e0db5-3e3a-4b2d-809a-5388f3650691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:20.455872 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e0db5-3e3a-4b2d-809a-5388f3650691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:20.456549 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e0db5-3e3a-4b2d-809a-5388f3650691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:20.457116 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e0db5-3e3a-4b2d-809a-5388f3650691 admin admin] PATH_INFO: `/v3/users/b690cc866cf44fb99f209782bb7439ae` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:20.459502 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e0db5-3e3a-4b2d-809a-5388f3650691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b690cc866cf44fb99f209782bb7439ae)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:20.461846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e0db5-3e3a-4b2d-809a-5388f3650691 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:20.512869 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:20.601381 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-163e0db5-3e3a-4b2d-809a-5388f3650691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b690cc866cf44fb99f209782bb7439ae'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:20.625768 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-163e0db5-3e3a-4b2d-809a-5388f3650691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b690cc866cf44fb99f209782bb7439ae'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:20.642720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-163e0db5-3e3a-4b2d-809a-5388f3650691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b690cc866cf44fb99f209782bb7439ae'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:20.644122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-163e0db5-3e3a-4b2d-809a-5388f3650691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b690cc866cf44fb99f209782bb7439ae'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:20.656484 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-163e0db5-3e3a-4b2d-809a-5388f3650691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b690cc866cf44fb99f209782bb7439ae'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:20.665694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1153/2357] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:48:20 2022] DELETE /identity/v3/users/b690cc866cf44fb99f209782bb7439ae => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:20.670861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe934d-b6d3-4362-a548-00073416c90b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:20.671438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe934d-b6d3-4362-a548-00073416c90b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:20.672827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe934d-b6d3-4362-a548-00073416c90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:20.692823 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-73fe934d-b6d3-4362-a548-00073416c90b None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:20.694705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-73fe934d-b6d3-4362-a548-00073416c90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:20.795066 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1205/2358] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:20.808923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e0db5-3e3a-4b2d-809a-5388f3650691 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:20.812653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-163e0db5-3e3a-4b2d-809a-5388f3650691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:20.934528 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e0db5-3e3a-4b2d-809a-5388f3650691 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:20.939477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cd795-ed8b-434d-9235-951973ba9ea2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:20.940867 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cd795-ed8b-434d-9235-951973ba9ea2 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:20.941440 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cd795-ed8b-434d-9235-951973ba9ea2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:20.941885 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cd795-ed8b-434d-9235-951973ba9ea2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:20.953847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1cd795-ed8b-434d-9235-951973ba9ea2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:20.956210 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1cd795-ed8b-434d-9235-951973ba9ea2 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:20.969302 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1154/2359] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:21.546678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe934d-b6d3-4362-a548-00073416c90b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:21.549551 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-73fe934d-b6d3-4362-a548-00073416c90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:21.580521 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cd795-ed8b-434d-9235-951973ba9ea2 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:21.582181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-dc1cd795-ed8b-434d-9235-951973ba9ea2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:21.586259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe934d-b6d3-4362-a548-00073416c90b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:21.605017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cd795-ed8b-434d-9235-951973ba9ea2 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:21.611561 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ea319-2fbb-4e04-b41e-c24299cb7682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:21.612965 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ea319-2fbb-4e04-b41e-c24299cb7682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:21.616008 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ea319-2fbb-4e04-b41e-c24299cb7682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:21.617259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ea319-2fbb-4e04-b41e-c24299cb7682 admin admin] PATH_INFO: `/v3/users/c069545cf93d4fdcb89accf145441303` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:21.618762 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca1566-1df3-44fa-a73f-3dd306e872c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:21.619904 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca1566-1df3-44fa-a73f-3dd306e872c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:21.620289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca1566-1df3-44fa-a73f-3dd306e872c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:21.620998 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca1566-1df3-44fa-a73f-3dd306e872c2 admin admin] PATH_INFO: `/v3/projects/31e9a6cad6f54b8ea4080c864390afa9` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:21.623061 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ea319-2fbb-4e04-b41e-c24299cb7682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c069545cf93d4fdcb89accf145441303)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:21.624492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca1566-1df3-44fa-a73f-3dd306e872c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e9a6cad6f54b8ea4080c864390afa9)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:21.625754 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ea319-2fbb-4e04-b41e-c24299cb7682 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:21.628238 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca1566-1df3-44fa-a73f-3dd306e872c2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:21.727468 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6f5ea319-2fbb-4e04-b41e-c24299cb7682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c069545cf93d4fdcb89accf145441303'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:21.728824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6f5ea319-2fbb-4e04-b41e-c24299cb7682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c069545cf93d4fdcb89accf145441303'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:21.730704 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6f5ea319-2fbb-4e04-b41e-c24299cb7682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c069545cf93d4fdcb89accf145441303'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:21.731480 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-23ca1566-1df3-44fa-a73f-3dd306e872c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e9a6cad6f54b8ea4080c864390afa9', 'request_id': 'req-23ca1566-1df3-44fa-a73f-3dd306e872c2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:21.736954 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:21.753385 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-23ca1566-1df3-44fa-a73f-3dd306e872c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e9a6cad6f54b8ea4080c864390afa9', 'request_id': 'req-23ca1566-1df3-44fa-a73f-3dd306e872c2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:21.758813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6f5ea319-2fbb-4e04-b41e-c24299cb7682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c069545cf93d4fdcb89accf145441303'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:21.767975 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6f5ea319-2fbb-4e04-b41e-c24299cb7682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c069545cf93d4fdcb89accf145441303'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:21.774407 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-23ca1566-1df3-44fa-a73f-3dd306e872c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e9a6cad6f54b8ea4080c864390afa9', 'request_id': 'req-23ca1566-1df3-44fa-a73f-3dd306e872c2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:21.797447 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-23ca1566-1df3-44fa-a73f-3dd306e872c2 admin admin] The token cache is being invalidate because project 31e9a6cad6f54b8ea4080c864390afa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:48:21.798248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-23ca1566-1df3-44fa-a73f-3dd306e872c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:21.809647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1206/2360] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:48:21 2022] DELETE /identity/v3/users/c069545cf93d4fdcb89accf145441303 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:21.812754 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-23ca1566-1df3-44fa-a73f-3dd306e872c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:21.814710 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a72f3-7869-4c94-8954-3f1dd733238c None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:21.815004 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a72f3-7869-4c94-8954-3f1dd733238c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:21.815279 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a72f3-7869-4c94-8954-3f1dd733238c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:21.820187 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1155/2361] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:48:21 2022] DELETE /identity/v3/projects/31e9a6cad6f54b8ea4080c864390afa9 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:48:21.822162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cc224-7fa2-4e5f-ac53-1d7f6645dceb None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:21.822162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cc224-7fa2-4e5f-ac53-1d7f6645dceb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:21.823572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cc224-7fa2-4e5f-ac53-1d7f6645dceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:21.874861 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-c60cc224-7fa2-4e5f-ac53-1d7f6645dceb None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:21.876507 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c60cc224-7fa2-4e5f-ac53-1d7f6645dceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:21.895288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-e83a72f3-7869-4c94-8954-3f1dd733238c None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:21.896571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e83a72f3-7869-4c94-8954-3f1dd733238c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:21.911792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:22.043763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1156/2362] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:22.050130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1207/2363] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:22.065091 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cc224-7fa2-4e5f-ac53-1d7f6645dceb None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:22.066575 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c60cc224-7fa2-4e5f-ac53-1d7f6645dceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:22.131160 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7f37d-d0ed-43a7-a5c7-d666ad246c7c None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:22.131553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7f37d-d0ed-43a7-a5c7-d666ad246c7c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:22.131751 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7f37d-d0ed-43a7-a5c7-d666ad246c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:22.175604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-17a7f37d-d0ed-43a7-a5c7-d666ad246c7c None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:22.175604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-17a7f37d-d0ed-43a7-a5c7-d666ad246c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:22.187775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1208/2364] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:48:22.233385 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7f37d-d0ed-43a7-a5c7-d666ad246c7c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:22.245554 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cc224-7fa2-4e5f-ac53-1d7f6645dceb None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:22.255782 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7f37d-d0ed-43a7-a5c7-d666ad246c7c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:22.258500 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908bed2-0f24-486a-8232-d95d711bb539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:22.259957 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908bed2-0f24-486a-8232-d95d711bb539 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:22.260973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908bed2-0f24-486a-8232-d95d711bb539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:22.261428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908bed2-0f24-486a-8232-d95d711bb539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:22.266676 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a513182-ab52-4a78-880b-d02bcabe26e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:22.267362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a513182-ab52-4a78-880b-d02bcabe26e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:22.267645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a513182-ab52-4a78-880b-d02bcabe26e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:22.267921 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a513182-ab52-4a78-880b-d02bcabe26e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:22.268653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a513182-ab52-4a78-880b-d02bcabe26e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:48:22.268961 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a513182-ab52-4a78-880b-d02bcabe26e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:22.270622 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a513182-ab52-4a78-880b-d02bcabe26e6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:22.273878 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908bed2-0f24-486a-8232-d95d711bb539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:22.275559 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908bed2-0f24-486a-8232-d95d711bb539 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:22.287448 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1157/2365] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:22.323622 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1209/2366] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:48:22.324166 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908bed2-0f24-486a-8232-d95d711bb539 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:22.336099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908bed2-0f24-486a-8232-d95d711bb539 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:22.347389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab71a7d-6ddd-4aab-b25f-c1effdacf2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:22.348076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab71a7d-6ddd-4aab-b25f-c1effdacf2bb admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:22.348361 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab71a7d-6ddd-4aab-b25f-c1effdacf2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:22.348639 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab71a7d-6ddd-4aab-b25f-c1effdacf2bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:22.349354 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab71a7d-6ddd-4aab-b25f-c1effdacf2bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:22.351326 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab71a7d-6ddd-4aab-b25f-c1effdacf2bb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:22.408557 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1158/2367] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:48:22 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:48:22.409195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a513182-ab52-4a78-880b-d02bcabe26e6 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:22.423424 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a513182-ab52-4a78-880b-d02bcabe26e6 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:22.619249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2bfc9-b3ee-44c3-a5b0-2ab78a072790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:22.621545 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2bfc9-b3ee-44c3-a5b0-2ab78a072790 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:22.622631 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2bfc9-b3ee-44c3-a5b0-2ab78a072790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:22.622929 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2bfc9-b3ee-44c3-a5b0-2ab78a072790 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:22.624958 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2bfc9-b3ee-44c3-a5b0-2ab78a072790 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:22.630768 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2bfc9-b3ee-44c3-a5b0-2ab78a072790 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:22.671454 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1210/2368] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:48:22 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:22.677697 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab71a7d-6ddd-4aab-b25f-c1effdacf2bb admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:22.709487 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab71a7d-6ddd-4aab-b25f-c1effdacf2bb admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:22.715118 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548978fe-5fb7-465f-9fb6-1fa6557c894c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:22.716197 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548978fe-5fb7-465f-9fb6-1fa6557c894c admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:22.716417 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548978fe-5fb7-465f-9fb6-1fa6557c894c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:22.716983 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548978fe-5fb7-465f-9fb6-1fa6557c894c admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:22.717888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548978fe-5fb7-465f-9fb6-1fa6557c894c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:22.719629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548978fe-5fb7-465f-9fb6-1fa6557c894c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:22.741143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1159/2369] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:22.742620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2bfc9-b3ee-44c3-a5b0-2ab78a072790 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:22.756298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2bfc9-b3ee-44c3-a5b0-2ab78a072790 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:22.775124 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30485534-9a2a-4849-b1bb-e1fc5d0c1178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:22.776400 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30485534-9a2a-4849-b1bb-e1fc5d0c1178 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:22.777393 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30485534-9a2a-4849-b1bb-e1fc5d0c1178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:22.777783 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30485534-9a2a-4849-b1bb-e1fc5d0c1178 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:22.778696 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30485534-9a2a-4849-b1bb-e1fc5d0c1178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:22.782957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30485534-9a2a-4849-b1bb-e1fc5d0c1178 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:22.840302 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548978fe-5fb7-465f-9fb6-1fa6557c894c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:22.843100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-548978fe-5fb7-465f-9fb6-1fa6557c894c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:22.849672 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1211/2370] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:22.853280 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30485534-9a2a-4849-b1bb-e1fc5d0c1178 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:22.883827 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548978fe-5fb7-465f-9fb6-1fa6557c894c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:22.898072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30485534-9a2a-4849-b1bb-e1fc5d0c1178 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:22.911852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4527cf0-1b9e-453b-bd42-88243713766b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:22.913046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4527cf0-1b9e-453b-bd42-88243713766b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:22.913583 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4527cf0-1b9e-453b-bd42-88243713766b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:22.914282 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4527cf0-1b9e-453b-bd42-88243713766b admin admin] PATH_INFO: `/v3/projects/31b63a8437804ab79bd2d3b9d858ed8b` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:22.915681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a11bba-d296-4416-83ef-8a03d3f18dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:22.917260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4527cf0-1b9e-453b-bd42-88243713766b admin admin] RBAC: Authorizing `identity:delete_project(project_id=31b63a8437804ab79bd2d3b9d858ed8b)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:22.921418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a11bba-d296-4416-83ef-8a03d3f18dae admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:22.921717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a11bba-d296-4416-83ef-8a03d3f18dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:22.922026 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a11bba-d296-4416-83ef-8a03d3f18dae admin admin] PATH_INFO: `/v3/projects/70562708174846e9bb7b1b128a629a98/users/ab68f09ccf4d4ac9901d4ed792b551c0/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:22.923890 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4527cf0-1b9e-453b-bd42-88243713766b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:23.000413 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a11bba-d296-4416-83ef-8a03d3f18dae admin admin] RBAC: Authorizing `identity:create_grant(project_id=70562708174846e9bb7b1b128a629a98, user_id=ab68f09ccf4d4ac9901d4ed792b551c0, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:23.031083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a11bba-d296-4416-83ef-8a03d3f18dae admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:23.057990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1212/2371] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:22 2022] PUT /identity/v3/projects/70562708174846e9bb7b1b128a629a98/users/ab68f09ccf4d4ac9901d4ed792b551c0/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:23.065283 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee43550-df08-480b-9416-802120368647 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:23.065581 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee43550-df08-480b-9416-802120368647 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:23.066242 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee43550-df08-480b-9416-802120368647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:23.079108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:23.103516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1ee43550-df08-480b-9416-802120368647 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:23.105406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1ee43550-df08-480b-9416-802120368647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:23.131033 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e4527cf0-1b9e-453b-bd42-88243713766b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31b63a8437804ab79bd2d3b9d858ed8b', 'request_id': 'req-e4527cf0-1b9e-453b-bd42-88243713766b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:23.152823 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e4527cf0-1b9e-453b-bd42-88243713766b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31b63a8437804ab79bd2d3b9d858ed8b', 'request_id': 'req-e4527cf0-1b9e-453b-bd42-88243713766b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:23.173795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e4527cf0-1b9e-453b-bd42-88243713766b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31b63a8437804ab79bd2d3b9d858ed8b', 'request_id': 'req-e4527cf0-1b9e-453b-bd42-88243713766b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:23.196040 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e4527cf0-1b9e-453b-bd42-88243713766b admin admin] The token cache is being invalidate because project 31b63a8437804ab79bd2d3b9d858ed8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:48:23.196524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e4527cf0-1b9e-453b-bd42-88243713766b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:23.198745 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-e4527cf0-1b9e-453b-bd42-88243713766b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:23.208301 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1160/2372] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:48:22 2022] DELETE /identity/v3/projects/31b63a8437804ab79bd2d3b9d858ed8b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:48:23.211707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4527cf0-1b9e-453b-bd42-88243713766b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:23.214139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e4527cf0-1b9e-453b-bd42-88243713766b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:23.250275 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:23.342496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1213/2373] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:23.394213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee43550-df08-480b-9416-802120368647 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:23.395786 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1ee43550-df08-480b-9416-802120368647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:23.414269 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4527cf0-1b9e-453b-bd42-88243713766b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:23.444995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dc30f-aad1-4a79-811b-b78e4a2a810b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:23.447738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dc30f-aad1-4a79-811b-b78e4a2a810b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:23.448259 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dc30f-aad1-4a79-811b-b78e4a2a810b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:23.449051 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dc30f-aad1-4a79-811b-b78e4a2a810b admin admin] PATH_INFO: `/v3/users/b6206c760eef4056a561212b38ab097d` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:23.456179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40dc30f-aad1-4a79-811b-b78e4a2a810b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6206c760eef4056a561212b38ab097d)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:23.460985 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40dc30f-aad1-4a79-811b-b78e4a2a810b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:23.533420 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee43550-df08-480b-9416-802120368647 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:23.545107 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963eeeb4-0315-4f89-8c39-4f8254450c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:23.546518 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963eeeb4-0315-4f89-8c39-4f8254450c07 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:23.547046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963eeeb4-0315-4f89-8c39-4f8254450c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:23.547380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963eeeb4-0315-4f89-8c39-4f8254450c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:23.561375 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963eeeb4-0315-4f89-8c39-4f8254450c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:23.563057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963eeeb4-0315-4f89-8c39-4f8254450c07 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:23.575648 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1214/2374] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:48:23.575837 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c40dc30f-aad1-4a79-811b-b78e4a2a810b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6206c760eef4056a561212b38ab097d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:23.607550 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c40dc30f-aad1-4a79-811b-b78e4a2a810b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6206c760eef4056a561212b38ab097d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:23.638100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c40dc30f-aad1-4a79-811b-b78e4a2a810b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6206c760eef4056a561212b38ab097d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:23.640583 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c40dc30f-aad1-4a79-811b-b78e4a2a810b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6206c760eef4056a561212b38ab097d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:23.661406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c40dc30f-aad1-4a79-811b-b78e4a2a810b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6206c760eef4056a561212b38ab097d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:23.684353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1161/2375] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:48:23 2022] DELETE /identity/v3/users/b6206c760eef4056a561212b38ab097d => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:48:24.164705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963eeeb4-0315-4f89-8c39-4f8254450c07 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:24.190602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963eeeb4-0315-4f89-8c39-4f8254450c07 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:24.355235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8a526-1878-4966-9f60-030a152327c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:24.357225 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8a526-1878-4966-9f60-030a152327c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:24.357927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8a526-1878-4966-9f60-030a152327c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:24.358383 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8a526-1878-4966-9f60-030a152327c0 admin admin] PATH_INFO: `/v3/projects/a76a451c5ede4e4a80b349ef10ec075e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:24.360133 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8a526-1878-4966-9f60-030a152327c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a76a451c5ede4e4a80b349ef10ec075e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:24.363894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8a526-1878-4966-9f60-030a152327c0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:24.499546 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9dc8a526-1878-4966-9f60-030a152327c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a76a451c5ede4e4a80b349ef10ec075e', 'request_id': 'req-9dc8a526-1878-4966-9f60-030a152327c0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:24.511478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:24.530205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9dc8a526-1878-4966-9f60-030a152327c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a76a451c5ede4e4a80b349ef10ec075e', 'request_id': 'req-9dc8a526-1878-4966-9f60-030a152327c0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:24.559340 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9dc8a526-1878-4966-9f60-030a152327c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a76a451c5ede4e4a80b349ef10ec075e', 'request_id': 'req-9dc8a526-1878-4966-9f60-030a152327c0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:24.603046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9dc8a526-1878-4966-9f60-030a152327c0 admin admin] The token cache is being invalidate because project a76a451c5ede4e4a80b349ef10ec075e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:48:24.603602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9dc8a526-1878-4966-9f60-030a152327c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:24.605050 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9dc8a526-1878-4966-9f60-030a152327c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:24.618695 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dc30f-aad1-4a79-811b-b78e4a2a810b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:24.620386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1215/2376] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:48:24 2022] DELETE /identity/v3/projects/a76a451c5ede4e4a80b349ef10ec075e => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:24.621259 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c40dc30f-aad1-4a79-811b-b78e4a2a810b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:24.648351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dc30f-aad1-4a79-811b-b78e4a2a810b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:24.805546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70d77e-4cde-4db2-92c3-b0fc67146adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:24.807973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70d77e-4cde-4db2-92c3-b0fc67146adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:24.808320 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70d77e-4cde-4db2-92c3-b0fc67146adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:24.808902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70d77e-4cde-4db2-92c3-b0fc67146adc admin admin] PATH_INFO: `/v3/users/72dc864700234a6d8af419366dc83f49` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:24.811855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70d77e-4cde-4db2-92c3-b0fc67146adc admin admin] RBAC: Authorizing `identity:delete_user(user_id=72dc864700234a6d8af419366dc83f49)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:24.816703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70d77e-4cde-4db2-92c3-b0fc67146adc admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:24.908943 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8a526-1878-4966-9f60-030a152327c0 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:24.911036 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9dc8a526-1878-4966-9f60-030a152327c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:24.914430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:24.937389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6c70d77e-4cde-4db2-92c3-b0fc67146adc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72dc864700234a6d8af419366dc83f49'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:24.958208 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6c70d77e-4cde-4db2-92c3-b0fc67146adc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72dc864700234a6d8af419366dc83f49'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:24.970321 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6c70d77e-4cde-4db2-92c3-b0fc67146adc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72dc864700234a6d8af419366dc83f49'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:24.971213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6c70d77e-4cde-4db2-92c3-b0fc67146adc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72dc864700234a6d8af419366dc83f49'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:24.989430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6c70d77e-4cde-4db2-92c3-b0fc67146adc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72dc864700234a6d8af419366dc83f49'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:25.014594 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1162/2377] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:48:24 2022] DELETE /identity/v3/users/72dc864700234a6d8af419366dc83f49 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:25.104668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8a526-1878-4966-9f60-030a152327c0 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:25.226807 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88cb61-dc05-495b-ae3e-2d24e2fc1284 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1308deea1ea4629a1ac22d4b903ad4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:25.229790 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88cb61-dc05-495b-ae3e-2d24e2fc1284 service placement] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:25.233332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88cb61-dc05-495b-ae3e-2d24e2fc1284 service placement] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:25.238631 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88cb61-dc05-495b-ae3e-2d24e2fc1284 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:25.241891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1a88cb61-dc05-495b-ae3e-2d24e2fc1284 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:25.256732 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88cb61-dc05-495b-ae3e-2d24e2fc1284 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:25.259156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88cb61-dc05-495b-ae3e-2d24e2fc1284 service placement] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:25.306064 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1216/2378] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:25.917019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70d77e-4cde-4db2-92c3-b0fc67146adc admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:25.919093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6c70d77e-4cde-4db2-92c3-b0fc67146adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:26.098808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70d77e-4cde-4db2-92c3-b0fc67146adc admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:26.113116 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1b0d1-25fd-4113-ba3f-67090154f52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:26.116737 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1b0d1-25fd-4113-ba3f-67090154f52d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:26.118834 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1b0d1-25fd-4113-ba3f-67090154f52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:26.120777 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1b0d1-25fd-4113-ba3f-67090154f52d admin admin] PATH_INFO: `/v3/projects/a356ccce83e3442bad4ba30c50cf28e7` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:26.122384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1b0d1-25fd-4113-ba3f-67090154f52d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a356ccce83e3442bad4ba30c50cf28e7)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:26.132333 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1b0d1-25fd-4113-ba3f-67090154f52d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:26.227065 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-cbd1b0d1-25fd-4113-ba3f-67090154f52d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a356ccce83e3442bad4ba30c50cf28e7', 'request_id': 'req-cbd1b0d1-25fd-4113-ba3f-67090154f52d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:26.246863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-cbd1b0d1-25fd-4113-ba3f-67090154f52d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a356ccce83e3442bad4ba30c50cf28e7', 'request_id': 'req-cbd1b0d1-25fd-4113-ba3f-67090154f52d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:26.259186 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-cbd1b0d1-25fd-4113-ba3f-67090154f52d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a356ccce83e3442bad4ba30c50cf28e7', 'request_id': 'req-cbd1b0d1-25fd-4113-ba3f-67090154f52d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:26.297408 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-cbd1b0d1-25fd-4113-ba3f-67090154f52d admin admin] The token cache is being invalidate because project a356ccce83e3442bad4ba30c50cf28e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:48:26.298246 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-cbd1b0d1-25fd-4113-ba3f-67090154f52d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:26.301656 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-cbd1b0d1-25fd-4113-ba3f-67090154f52d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:26.304262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1163/2379] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:48:25 2022] DELETE /identity/v3/projects/a356ccce83e3442bad4ba30c50cf28e7 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:26.310407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fe743-bd37-4761-8d7a-be34d812410f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:26.311089 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fe743-bd37-4761-8d7a-be34d812410f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:26.311089 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fe743-bd37-4761-8d7a-be34d812410f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:26.332204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-508fe743-bd37-4761-8d7a-be34d812410f None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:26.333893 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-508fe743-bd37-4761-8d7a-be34d812410f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:26.440490 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1217/2380] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:26.450426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be895c5-2362-4ebf-a03a-f3ae1aa167de None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:26.451029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be895c5-2362-4ebf-a03a-f3ae1aa167de None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:26.451497 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be895c5-2362-4ebf-a03a-f3ae1aa167de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:26.494890 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9be895c5-2362-4ebf-a03a-f3ae1aa167de None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:26.502184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9be895c5-2362-4ebf-a03a-f3ae1aa167de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:26.531675 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1164/2381] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:26.532853 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508fe743-bd37-4761-8d7a-be34d812410f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:26.550514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508fe743-bd37-4761-8d7a-be34d812410f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:26.559593 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6827092-960d-476f-aab3-d784fe605639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:26.560562 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6827092-960d-476f-aab3-d784fe605639 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:26.561022 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6827092-960d-476f-aab3-d784fe605639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:26.561456 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6827092-960d-476f-aab3-d784fe605639 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:26.562442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6827092-960d-476f-aab3-d784fe605639 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:48:26.562933 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6827092-960d-476f-aab3-d784fe605639 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:26.565205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6827092-960d-476f-aab3-d784fe605639 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:26.585304 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:26.605391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1218/2382] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:48:26.612201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be895c5-2362-4ebf-a03a-f3ae1aa167de None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:26.634090 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be895c5-2362-4ebf-a03a-f3ae1aa167de None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:26.646051 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15811e7e-b576-4966-9962-b43f80385a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:26.649993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15811e7e-b576-4966-9962-b43f80385a81 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:26.649993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15811e7e-b576-4966-9962-b43f80385a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:26.649993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15811e7e-b576-4966-9962-b43f80385a81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:26.649993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15811e7e-b576-4966-9962-b43f80385a81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:26.660837 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15811e7e-b576-4966-9962-b43f80385a81 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:26.716146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1165/2383] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:48:26 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:26.716505 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6827092-960d-476f-aab3-d784fe605639 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:26.746541 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6827092-960d-476f-aab3-d784fe605639 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:26.890132 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decee45-19a9-4e15-8c16-7fe7abe07a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:26.891624 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decee45-19a9-4e15-8c16-7fe7abe07a18 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:26.892288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decee45-19a9-4e15-8c16-7fe7abe07a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:26.893460 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decee45-19a9-4e15-8c16-7fe7abe07a18 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:26.895783 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decee45-19a9-4e15-8c16-7fe7abe07a18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:26.900828 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decee45-19a9-4e15-8c16-7fe7abe07a18 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:26.961004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15811e7e-b576-4966-9962-b43f80385a81 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:26.977081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1219/2384] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:48:26 2022] POST /identity/v3/users => generated 287 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:48:27.008876 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15811e7e-b576-4966-9962-b43f80385a81 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:27.033640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d558c39-0674-4002-b76d-63598e865dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:27.034676 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d558c39-0674-4002-b76d-63598e865dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:27.034958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d558c39-0674-4002-b76d-63598e865dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:27.035304 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d558c39-0674-4002-b76d-63598e865dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:27.036038 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d558c39-0674-4002-b76d-63598e865dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:27.038957 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d558c39-0674-4002-b76d-63598e865dd6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:27.099110 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1166/2385] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:27.102349 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decee45-19a9-4e15-8c16-7fe7abe07a18 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:27.127921 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decee45-19a9-4e15-8c16-7fe7abe07a18 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:27.134211 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677e257-6c3b-4a75-bd9d-acc344049518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:27.135537 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677e257-6c3b-4a75-bd9d-acc344049518 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:27.136195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677e257-6c3b-4a75-bd9d-acc344049518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:27.136685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677e257-6c3b-4a75-bd9d-acc344049518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:27.137663 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677e257-6c3b-4a75-bd9d-acc344049518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:27.139830 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677e257-6c3b-4a75-bd9d-acc344049518 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:27.187897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d558c39-0674-4002-b76d-63598e865dd6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:27.189034 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1220/2386] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:27.194618 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:27.224539 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d558c39-0674-4002-b76d-63598e865dd6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:27.240122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2b891-f8c4-4b36-9e52-1f046b8c42d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:27.241196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2b891-f8c4-4b36-9e52-1f046b8c42d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:27.241589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2b891-f8c4-4b36-9e52-1f046b8c42d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:27.241987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2b891-f8c4-4b36-9e52-1f046b8c42d8 admin admin] PATH_INFO: `/v3/projects/8d9974eddb1945bcb68327194b261254/users/e86d71c0fd3f4e08b2f525afee8bf556/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:27.280094 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2b891-f8c4-4b36-9e52-1f046b8c42d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d9974eddb1945bcb68327194b261254, user_id=e86d71c0fd3f4e08b2f525afee8bf556, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:27.282751 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2b891-f8c4-4b36-9e52-1f046b8c42d8 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:27.322802 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1167/2387] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:27 2022] PUT /identity/v3/projects/8d9974eddb1945bcb68327194b261254/users/e86d71c0fd3f4e08b2f525afee8bf556/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:27.431960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6fd86-3386-4260-a721-8309c2c97ff0 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:27.433473 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6fd86-3386-4260-a721-8309c2c97ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:27.434223 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6fd86-3386-4260-a721-8309c2c97ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:27.541194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-fef6fd86-3386-4260-a721-8309c2c97ff0 None None] MFA Rules not processed for user `e86d71c0fd3f4e08b2f525afee8bf556`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:27.544566 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fef6fd86-3386-4260-a721-8309c2c97ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:27.759546 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1221/2388] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:27.768098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2b891-f8c4-4b36-9e52-1f046b8c42d8 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:27.769995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-96d2b891-f8c4-4b36-9e52-1f046b8c42d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:27.995749 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2b891-f8c4-4b36-9e52-1f046b8c42d8 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:28.002893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e93bcf-38bf-4bd4-a434-a3487fd2311c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:28.003776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e93bcf-38bf-4bd4-a434-a3487fd2311c service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:28.004216 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e93bcf-38bf-4bd4-a434-a3487fd2311c service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:28.006987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e93bcf-38bf-4bd4-a434-a3487fd2311c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:28.024515 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e93bcf-38bf-4bd4-a434-a3487fd2311c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:28.030078 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e93bcf-38bf-4bd4-a434-a3487fd2311c service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:28.050503 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1168/2389] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:32.549908 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6fd86-3386-4260-a721-8309c2c97ff0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:32.552779 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fef6fd86-3386-4260-a721-8309c2c97ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:32.656402 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6fd86-3386-4260-a721-8309c2c97ff0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:32.663763 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedb961-8fb5-4ea0-9b7a-d67f7737f1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:32.665003 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedb961-8fb5-4ea0-9b7a-d67f7737f1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:32.665713 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedb961-8fb5-4ea0-9b7a-d67f7737f1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:32.674166 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedb961-8fb5-4ea0-9b7a-d67f7737f1f8 admin admin] PATH_INFO: `/v3/users/bb11db914a57439bada24b5f755a143a` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:32.674166 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedb961-8fb5-4ea0-9b7a-d67f7737f1f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb11db914a57439bada24b5f755a143a)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:32.674166 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedb961-8fb5-4ea0-9b7a-d67f7737f1f8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:32.681194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:32.685211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c893335-615e-44d2-af72-7ee866380e07 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:32.686010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c893335-615e-44d2-af72-7ee866380e07 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:32.686010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c893335-615e-44d2-af72-7ee866380e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:32.749308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4cedb961-8fb5-4ea0-9b7a-d67f7737f1f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb11db914a57439bada24b5f755a143a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:32.751668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4cedb961-8fb5-4ea0-9b7a-d67f7737f1f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb11db914a57439bada24b5f755a143a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:32.752697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4cedb961-8fb5-4ea0-9b7a-d67f7737f1f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb11db914a57439bada24b5f755a143a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:32.770786 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0c893335-615e-44d2-af72-7ee866380e07 None None] MFA Rules not processed for user `ab68f09ccf4d4ac9901d4ed792b551c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:32.771203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4cedb961-8fb5-4ea0-9b7a-d67f7737f1f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb11db914a57439bada24b5f755a143a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:32.772889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0c893335-615e-44d2-af72-7ee866380e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:32.781636 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4cedb961-8fb5-4ea0-9b7a-d67f7737f1f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb11db914a57439bada24b5f755a143a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:32.812650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1222/2390] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:48:32 2022] DELETE /identity/v3/users/bb11db914a57439bada24b5f755a143a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:32.924320 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1169/2391] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:32.928478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedb961-8fb5-4ea0-9b7a-d67f7737f1f8 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:32.930110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4cedb961-8fb5-4ea0-9b7a-d67f7737f1f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:33.030659 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedb961-8fb5-4ea0-9b7a-d67f7737f1f8 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:33.039526 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b312e-2a35-4ae1-b918-81629e63dcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:33.040983 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b312e-2a35-4ae1-b918-81629e63dcf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:33.041903 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b312e-2a35-4ae1-b918-81629e63dcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:33.042523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b312e-2a35-4ae1-b918-81629e63dcf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:33.044010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b312e-2a35-4ae1-b918-81629e63dcf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:33.047976 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b312e-2a35-4ae1-b918-81629e63dcf2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:33.106193 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1223/2392] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:48:32 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:33.107268 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c893335-615e-44d2-af72-7ee866380e07 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:33.125940 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c893335-615e-44d2-af72-7ee866380e07 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:33.210038 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2330243-3346-424d-b47a-a714c0eea8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:33.211250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2330243-3346-424d-b47a-a714c0eea8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:33.211671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2330243-3346-424d-b47a-a714c0eea8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:33.211989 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2330243-3346-424d-b47a-a714c0eea8c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:33.212809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2330243-3346-424d-b47a-a714c0eea8c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:33.215852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2330243-3346-424d-b47a-a714c0eea8c3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:33.236586 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:33.254411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1170/2393] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:48:33 2022] POST /identity/v3/users => generated 299 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:48:33.262485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b312e-2a35-4ae1-b918-81629e63dcf2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:33.264716 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a25b312e-2a35-4ae1-b918-81629e63dcf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:33.271265 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2330243-3346-424d-b47a-a714c0eea8c3 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:33.287429 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2330243-3346-424d-b47a-a714c0eea8c3 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:33.287813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b312e-2a35-4ae1-b918-81629e63dcf2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:33.296281 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45baf4d3-fe7d-484d-bb99-7678c29edaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:33.296965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45baf4d3-fe7d-484d-bb99-7678c29edaae admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:33.297246 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45baf4d3-fe7d-484d-bb99-7678c29edaae admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:33.297603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45baf4d3-fe7d-484d-bb99-7678c29edaae admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:33.298370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45baf4d3-fe7d-484d-bb99-7678c29edaae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:33.298774 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe521fd-5c65-4193-a7f7-3a716301ca4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:33.300904 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45baf4d3-fe7d-484d-bb99-7678c29edaae admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:33.302510 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe521fd-5c65-4193-a7f7-3a716301ca4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:33.303151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe521fd-5c65-4193-a7f7-3a716301ca4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:33.303954 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe521fd-5c65-4193-a7f7-3a716301ca4c admin admin] PATH_INFO: `/v3/projects/edae8cd268da40a3b8e2affe277bc56e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:33.305420 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe521fd-5c65-4193-a7f7-3a716301ca4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=edae8cd268da40a3b8e2affe277bc56e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:33.308304 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe521fd-5c65-4193-a7f7-3a716301ca4c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:33.317052 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1171/2394] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:33.318040 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45baf4d3-fe7d-484d-bb99-7678c29edaae admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:33.329601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45baf4d3-fe7d-484d-bb99-7678c29edaae admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:33.336370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab51146-975e-46fe-b46d-6811bb8f4fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:33.337434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab51146-975e-46fe-b46d-6811bb8f4fac admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:33.337723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab51146-975e-46fe-b46d-6811bb8f4fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:33.338187 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab51146-975e-46fe-b46d-6811bb8f4fac admin admin] PATH_INFO: `/v3/projects/dd37dadb214840cb87235a9b7e5acbff/users/90c83dec7921400ca1ec2e2cdbb38913/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:33.365723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab51146-975e-46fe-b46d-6811bb8f4fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd37dadb214840cb87235a9b7e5acbff, user_id=90c83dec7921400ca1ec2e2cdbb38913, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:33.369213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab51146-975e-46fe-b46d-6811bb8f4fac admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:33.402370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1172/2395] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:33 2022] PUT /identity/v3/projects/dd37dadb214840cb87235a9b7e5acbff/users/90c83dec7921400ca1ec2e2cdbb38913/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:33.406302 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab51146-975e-46fe-b46d-6811bb8f4fac admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:33.464261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fbe521fd-5c65-4193-a7f7-3a716301ca4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edae8cd268da40a3b8e2affe277bc56e', 'request_id': 'req-fbe521fd-5c65-4193-a7f7-3a716301ca4c'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:33.475131 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fbe521fd-5c65-4193-a7f7-3a716301ca4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edae8cd268da40a3b8e2affe277bc56e', 'request_id': 'req-fbe521fd-5c65-4193-a7f7-3a716301ca4c'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:33.486535 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fbe521fd-5c65-4193-a7f7-3a716301ca4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edae8cd268da40a3b8e2affe277bc56e', 'request_id': 'req-fbe521fd-5c65-4193-a7f7-3a716301ca4c'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:33.492112 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:33.506884 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fbe521fd-5c65-4193-a7f7-3a716301ca4c admin admin] The token cache is being invalidate because project edae8cd268da40a3b8e2affe277bc56e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:48:33.507169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fbe521fd-5c65-4193-a7f7-3a716301ca4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:33.507915 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fbe521fd-5c65-4193-a7f7-3a716301ca4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:33.510325 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1224/2396] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:48:33 2022] DELETE /identity/v3/projects/edae8cd268da40a3b8e2affe277bc56e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:33.514793 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe521fd-5c65-4193-a7f7-3a716301ca4c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:33.516540 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fbe521fd-5c65-4193-a7f7-3a716301ca4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:33.519118 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab51146-975e-46fe-b46d-6811bb8f4fac admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:33.647155 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447f180-4f2b-4a29-aa0c-e271781a697a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:33.647390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447f180-4f2b-4a29-aa0c-e271781a697a admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:33.647686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447f180-4f2b-4a29-aa0c-e271781a697a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:33.647976 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447f180-4f2b-4a29-aa0c-e271781a697a admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:33.649349 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447f180-4f2b-4a29-aa0c-e271781a697a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:33.652726 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447f180-4f2b-4a29-aa0c-e271781a697a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:33.660995 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe521fd-5c65-4193-a7f7-3a716301ca4c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:33.677757 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe9217-388e-4fab-aad8-6130cca0bb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:33.679081 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe9217-388e-4fab-aad8-6130cca0bb2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:33.692960 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1173/2397] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:33.694542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe9217-388e-4fab-aad8-6130cca0bb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:33.696587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe9217-388e-4fab-aad8-6130cca0bb2e admin admin] PATH_INFO: `/v3/users/ebc1b31030634b5eb17d3549959f5507` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:33.701200 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447f180-4f2b-4a29-aa0c-e271781a697a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:33.701551 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe9217-388e-4fab-aad8-6130cca0bb2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebc1b31030634b5eb17d3549959f5507)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:33.703803 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe9217-388e-4fab-aad8-6130cca0bb2e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:33.704859 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d447f180-4f2b-4a29-aa0c-e271781a697a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:33.756790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447f180-4f2b-4a29-aa0c-e271781a697a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:33.763635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898e8be-ab06-47c3-863a-58ccaf648f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:33.765907 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898e8be-ab06-47c3-863a-58ccaf648f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:33.765907 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898e8be-ab06-47c3-863a-58ccaf648f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:33.771931 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898e8be-ab06-47c3-863a-58ccaf648f0f admin admin] PATH_INFO: `/v3/projects/dd37dadb214840cb87235a9b7e5acbff/users/90c83dec7921400ca1ec2e2cdbb38913/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:33.779435 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898e8be-ab06-47c3-863a-58ccaf648f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd37dadb214840cb87235a9b7e5acbff, user_id=90c83dec7921400ca1ec2e2cdbb38913, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:33.783041 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898e8be-ab06-47c3-863a-58ccaf648f0f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:33.806480 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1174/2398] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:33 2022] PUT /identity/v3/projects/dd37dadb214840cb87235a9b7e5acbff/users/90c83dec7921400ca1ec2e2cdbb38913/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:33.841768 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c7fe9217-388e-4fab-aad8-6130cca0bb2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebc1b31030634b5eb17d3549959f5507'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:33.846390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c7fe9217-388e-4fab-aad8-6130cca0bb2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebc1b31030634b5eb17d3549959f5507'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:33.849991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c7fe9217-388e-4fab-aad8-6130cca0bb2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebc1b31030634b5eb17d3549959f5507'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:33.878291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c7fe9217-388e-4fab-aad8-6130cca0bb2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebc1b31030634b5eb17d3549959f5507'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:33.895095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c7fe9217-388e-4fab-aad8-6130cca0bb2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebc1b31030634b5eb17d3549959f5507'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:33.930080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1225/2399] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:48:33 2022] DELETE /identity/v3/users/ebc1b31030634b5eb17d3549959f5507 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:34.613322 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898e8be-ab06-47c3-863a-58ccaf648f0f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:34.615910 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9898e8be-ab06-47c3-863a-58ccaf648f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:34.797319 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:34.814640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898e8be-ab06-47c3-863a-58ccaf648f0f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:34.949198 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d055a4-10be-433a-a1c7-abd4fb976e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:34.950242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d055a4-10be-433a-a1c7-abd4fb976e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:34.950537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d055a4-10be-433a-a1c7-abd4fb976e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:34.951170 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d055a4-10be-433a-a1c7-abd4fb976e68 admin admin] PATH_INFO: `/v3/projects/29f0628d57db4d9c8c6408298f4af853` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:34.952594 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d055a4-10be-433a-a1c7-abd4fb976e68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29f0628d57db4d9c8c6408298f4af853)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:34.954661 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d055a4-10be-433a-a1c7-abd4fb976e68 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:35.021180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-09d055a4-10be-433a-a1c7-abd4fb976e68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29f0628d57db4d9c8c6408298f4af853', 'request_id': 'req-09d055a4-10be-433a-a1c7-abd4fb976e68'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:35.033731 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-09d055a4-10be-433a-a1c7-abd4fb976e68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29f0628d57db4d9c8c6408298f4af853', 'request_id': 'req-09d055a4-10be-433a-a1c7-abd4fb976e68'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:35.046709 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-09d055a4-10be-433a-a1c7-abd4fb976e68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29f0628d57db4d9c8c6408298f4af853', 'request_id': 'req-09d055a4-10be-433a-a1c7-abd4fb976e68'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:35.068401 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-09d055a4-10be-433a-a1c7-abd4fb976e68 admin admin] The token cache is being invalidate because project 29f0628d57db4d9c8c6408298f4af853 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:48:35.068803 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-09d055a4-10be-433a-a1c7-abd4fb976e68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:35.069927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-09d055a4-10be-433a-a1c7-abd4fb976e68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:35.086861 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1175/2400] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:48:34 2022] DELETE /identity/v3/projects/29f0628d57db4d9c8c6408298f4af853 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:39.503600 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7839117-10e5-48ed-ab73-658519e241d2 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:39.504184 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7839117-10e5-48ed-ab73-658519e241d2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:39.504184 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7839117-10e5-48ed-ab73-658519e241d2 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:39.505682 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1226/2401] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jul 12 13:48:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:48:39.512813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa1ddf-3338-4125-94cc-96864c4f4e95 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:39.513768 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa1ddf-3338-4125-94cc-96864c4f4e95 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:39.514496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa1ddf-3338-4125-94cc-96864c4f4e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:39.548433 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-92fa1ddf-3338-4125-94cc-96864c4f4e95 None None] MFA Rules not processed for user `0d3915cde0d04bcfb9b5ce8c89af1e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:39.551791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-92fa1ddf-3338-4125-94cc-96864c4f4e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:39.671152 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1176/2402] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jul 12 13:48:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:48:39.685183 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7839117-10e5-48ed-ab73-658519e241d2 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:39.687137 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a7839117-10e5-48ed-ab73-658519e241d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:39.809000 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7839117-10e5-48ed-ab73-658519e241d2 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:39.812843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329135da-77d6-4870-b3ca-1fcb89d931da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebb4da7b00c64fafbb3724bae25b2853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:39.813518 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329135da-77d6-4870-b3ca-1fcb89d931da service glance] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:39.813839 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329135da-77d6-4870-b3ca-1fcb89d931da service glance] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:39.814153 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329135da-77d6-4870-b3ca-1fcb89d931da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:39.867684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329135da-77d6-4870-b3ca-1fcb89d931da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:39.869247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329135da-77d6-4870-b3ca-1fcb89d931da service glance] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:39.918271 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1227/2403] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:40.075314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ff485-efdc-4050-8fce-5baac311f132 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:40.075688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ff485-efdc-4050-8fce-5baac311f132 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:40.076041 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ff485-efdc-4050-8fce-5baac311f132 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:40.078062 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1177/2404] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jul 12 13:48:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:48:40.088094 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c23f4d-d734-4163-8488-d91d5268c526 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:40.088418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c23f4d-d734-4163-8488-d91d5268c526 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:40.088706 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c23f4d-d734-4163-8488-d91d5268c526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:40.138044 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-52c23f4d-d734-4163-8488-d91d5268c526 None None] MFA Rules not processed for user `3e833722fe4e493e9ddc4b275632536f`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:40.139839 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-52c23f4d-d734-4163-8488-d91d5268c526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:40.185446 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:40.318515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1228/2405] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jul 12 13:48:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:40.345289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ff485-efdc-4050-8fce-5baac311f132 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:40.347685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-313ff485-efdc-4050-8fce-5baac311f132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:40.375297 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ff485-efdc-4050-8fce-5baac311f132 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:40.386635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9f23f-043f-405b-a4ed-3a8c0313f085 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:40.387540 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9f23f-043f-405b-a4ed-3a8c0313f085 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:40.387991 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9f23f-043f-405b-a4ed-3a8c0313f085 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:40.388452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9f23f-043f-405b-a4ed-3a8c0313f085 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:40.410316 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9f23f-043f-405b-a4ed-3a8c0313f085 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:40.411788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9f23f-043f-405b-a4ed-3a8c0313f085 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:40.433486 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1178/2406] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:40.553800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b17ae1-b64d-4b4d-910e-c9d9286197ed None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:40.554446 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b17ae1-b64d-4b4d-910e-c9d9286197ed None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:40.554675 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b17ae1-b64d-4b4d-910e-c9d9286197ed None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:40.556469 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1229/2407] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jul 12 13:48:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:48:40.567463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ee2aa-6fc6-451e-bdbf-0909d174a1ac None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:40.568404 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ee2aa-6fc6-451e-bdbf-0909d174a1ac None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:40.568872 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ee2aa-6fc6-451e-bdbf-0909d174a1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:40.607582 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-fa4ee2aa-6fc6-451e-bdbf-0909d174a1ac None None] MFA Rules not processed for user `0d3915cde0d04bcfb9b5ce8c89af1e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:40.608944 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fa4ee2aa-6fc6-451e-bdbf-0909d174a1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:40.624815 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1179/2408] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jul 12 13:48:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:48:40.637503 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b17ae1-b64d-4b4d-910e-c9d9286197ed None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:40.653700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b17ae1-b64d-4b4d-910e-c9d9286197ed None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:40.665707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c2a7a8-c21c-4450-92a3-c2fd4db1bc63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:40.666567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2a7a8-c21c-4450-92a3-c2fd4db1bc63 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:40.666924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2a7a8-c21c-4450-92a3-c2fd4db1bc63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:40.667301 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2a7a8-c21c-4450-92a3-c2fd4db1bc63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:40.682221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c2a7a8-c21c-4450-92a3-c2fd4db1bc63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:40.685179 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c2a7a8-c21c-4450-92a3-c2fd4db1bc63 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:40.703169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1230/2409] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:42.371176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ee2aa-6fc6-451e-bdbf-0909d174a1ac None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:42.372740 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fa4ee2aa-6fc6-451e-bdbf-0909d174a1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:42.380105 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:42.458143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ee2aa-6fc6-451e-bdbf-0909d174a1ac None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:42.464779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7b1ce-4776-4fa5-a0ab-476be6e51932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:42.466389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7b1ce-4776-4fa5-a0ab-476be6e51932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:42.466933 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7b1ce-4776-4fa5-a0ab-476be6e51932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:42.467431 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7b1ce-4776-4fa5-a0ab-476be6e51932 admin admin] PATH_INFO: `/v3/users/e86d71c0fd3f4e08b2f525afee8bf556` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:42.469312 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7b1ce-4776-4fa5-a0ab-476be6e51932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e86d71c0fd3f4e08b2f525afee8bf556)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:42.473138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7b1ce-4776-4fa5-a0ab-476be6e51932 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:42.546457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-44d7b1ce-4776-4fa5-a0ab-476be6e51932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e86d71c0fd3f4e08b2f525afee8bf556'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:42.566506 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-44d7b1ce-4776-4fa5-a0ab-476be6e51932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e86d71c0fd3f4e08b2f525afee8bf556'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:42.577356 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-44d7b1ce-4776-4fa5-a0ab-476be6e51932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e86d71c0fd3f4e08b2f525afee8bf556'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:42.581179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-44d7b1ce-4776-4fa5-a0ab-476be6e51932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e86d71c0fd3f4e08b2f525afee8bf556'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:42.594254 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-44d7b1ce-4776-4fa5-a0ab-476be6e51932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e86d71c0fd3f4e08b2f525afee8bf556'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:42.605703 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1180/2410] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:48:42 2022] DELETE /identity/v3/users/e86d71c0fd3f4e08b2f525afee8bf556 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:42.610498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f146db2-5b06-4746-a212-70684fcadf39 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:42.611028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f146db2-5b06-4746-a212-70684fcadf39 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:42.611137 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f146db2-5b06-4746-a212-70684fcadf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:42.654216 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7f146db2-5b06-4746-a212-70684fcadf39 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:42.655908 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7f146db2-5b06-4746-a212-70684fcadf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:42.772587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1231/2411] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:42.791077 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7b1ce-4776-4fa5-a0ab-476be6e51932 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:42.792565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-44d7b1ce-4776-4fa5-a0ab-476be6e51932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:42.879142 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7b1ce-4776-4fa5-a0ab-476be6e51932 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:42.884453 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8784c9-60af-466d-845a-afac2c83ff66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:42.885996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8784c9-60af-466d-845a-afac2c83ff66 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:42.886501 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8784c9-60af-466d-845a-afac2c83ff66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:42.887024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8784c9-60af-466d-845a-afac2c83ff66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:42.899637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8784c9-60af-466d-845a-afac2c83ff66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:42.902223 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8784c9-60af-466d-845a-afac2c83ff66 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:42.913322 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1181/2412] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:43.386268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f146db2-5b06-4746-a212-70684fcadf39 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:43.388731 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7f146db2-5b06-4746-a212-70684fcadf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:43.404731 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f146db2-5b06-4746-a212-70684fcadf39 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:43.410573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d10a61-09ad-46ff-9385-37a18650190b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:43.413491 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d10a61-09ad-46ff-9385-37a18650190b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:43.414014 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d10a61-09ad-46ff-9385-37a18650190b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:43.414444 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d10a61-09ad-46ff-9385-37a18650190b admin admin] PATH_INFO: `/v3/projects/8d9974eddb1945bcb68327194b261254` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:43.416078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d10a61-09ad-46ff-9385-37a18650190b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d9974eddb1945bcb68327194b261254)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:43.419574 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d10a61-09ad-46ff-9385-37a18650190b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:43.512850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-54d10a61-09ad-46ff-9385-37a18650190b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d9974eddb1945bcb68327194b261254', 'request_id': 'req-54d10a61-09ad-46ff-9385-37a18650190b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:43.526550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-54d10a61-09ad-46ff-9385-37a18650190b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d9974eddb1945bcb68327194b261254', 'request_id': 'req-54d10a61-09ad-46ff-9385-37a18650190b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:43.538690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-54d10a61-09ad-46ff-9385-37a18650190b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d9974eddb1945bcb68327194b261254', 'request_id': 'req-54d10a61-09ad-46ff-9385-37a18650190b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:43.556106 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-54d10a61-09ad-46ff-9385-37a18650190b admin admin] The token cache is being invalidate because project 8d9974eddb1945bcb68327194b261254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:48:43.557078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-54d10a61-09ad-46ff-9385-37a18650190b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:43.561467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-54d10a61-09ad-46ff-9385-37a18650190b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:48:43.563667 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1232/2413] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:48:43 2022] DELETE /identity/v3/projects/8d9974eddb1945bcb68327194b261254 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:43.570625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630f163-d1eb-4508-b744-02d501edb330 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:43.571054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630f163-d1eb-4508-b744-02d501edb330 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:43.571054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630f163-d1eb-4508-b744-02d501edb330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:43.604989 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b630f163-d1eb-4508-b744-02d501edb330 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:43.606560 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b630f163-d1eb-4508-b744-02d501edb330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:43.657811 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:43.686736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07521230-6f3e-43ad-bafd-16cca4124d55 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:43.687294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07521230-6f3e-43ad-bafd-16cca4124d55 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:43.687486 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07521230-6f3e-43ad-bafd-16cca4124d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:43.724095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-07521230-6f3e-43ad-bafd-16cca4124d55 None None] MFA Rules not processed for user `90c83dec7921400ca1ec2e2cdbb38913`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:43.725990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-07521230-6f3e-43ad-bafd-16cca4124d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:43.736527 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1182/2414] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:43.738424 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b500ea-b14c-4c40-b767-866c8320df85 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:43.738424 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b500ea-b14c-4c40-b767-866c8320df85 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:43.738424 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b500ea-b14c-4c40-b767-866c8320df85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:43.756117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:43.759599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-20b500ea-b14c-4c40-b767-866c8320df85 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:43.761419 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-20b500ea-b14c-4c40-b767-866c8320df85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:43.777781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1183/2415] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:43.779608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b500ea-b14c-4c40-b767-866c8320df85 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:43.790479 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b500ea-b14c-4c40-b767-866c8320df85 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:43.804150 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be408120-ffc7-40fa-9307-11998bdb1dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:43.804891 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be408120-ffc7-40fa-9307-11998bdb1dcf admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:43.805175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be408120-ffc7-40fa-9307-11998bdb1dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:43.805714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be408120-ffc7-40fa-9307-11998bdb1dcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:43.806206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be408120-ffc7-40fa-9307-11998bdb1dcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:48:43.807037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be408120-ffc7-40fa-9307-11998bdb1dcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:43.809136 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be408120-ffc7-40fa-9307-11998bdb1dcf admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:43.839391 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1184/2416] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:48:43.842615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be408120-ffc7-40fa-9307-11998bdb1dcf admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:43.863541 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be408120-ffc7-40fa-9307-11998bdb1dcf admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:43.868771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238daf9f-aed7-4f2e-b50c-94dff27e0c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:43.870498 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238daf9f-aed7-4f2e-b50c-94dff27e0c9f admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:43.870865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238daf9f-aed7-4f2e-b50c-94dff27e0c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:43.871764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238daf9f-aed7-4f2e-b50c-94dff27e0c9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:43.872575 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238daf9f-aed7-4f2e-b50c-94dff27e0c9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:43.878353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238daf9f-aed7-4f2e-b50c-94dff27e0c9f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:43.900676 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1233/2417] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:43.924090 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07521230-6f3e-43ad-bafd-16cca4124d55 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:43.924090 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-07521230-6f3e-43ad-bafd-16cca4124d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:43.945594 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1185/2418] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:48:43 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:43.946785 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238daf9f-aed7-4f2e-b50c-94dff27e0c9f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:44.052306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07521230-6f3e-43ad-bafd-16cca4124d55 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:44.138670 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238daf9f-aed7-4f2e-b50c-94dff27e0c9f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:44.152076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e0f4a-e582-45ea-a002-bd94fccb551b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:44.152853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e0f4a-e582-45ea-a002-bd94fccb551b admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:44.153182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e0f4a-e582-45ea-a002-bd94fccb551b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:44.153428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e0f4a-e582-45ea-a002-bd94fccb551b admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:44.154433 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e0f4a-e582-45ea-a002-bd94fccb551b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:44.160800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e646a0-0330-45a5-a265-70378997b556 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebb4da7b00c64fafbb3724bae25b2853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:44.161743 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e0f4a-e582-45ea-a002-bd94fccb551b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:44.162086 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e646a0-0330-45a5-a265-70378997b556 service glance] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:44.162435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e646a0-0330-45a5-a265-70378997b556 service glance] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:44.162806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e646a0-0330-45a5-a265-70378997b556 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:44.168629 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-98e646a0-0330-45a5-a265-70378997b556 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:44.199823 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1186/2419] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:48:43 2022] POST /identity/v3/users => generated 287 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:48:44.201377 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f5522-9109-494f-9ea6-4e158264aeba None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:44.201773 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f5522-9109-494f-9ea6-4e158264aeba None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:44.203005 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f5522-9109-494f-9ea6-4e158264aeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:44.238816 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-583f5522-9109-494f-9ea6-4e158264aeba None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:44.242970 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-583f5522-9109-494f-9ea6-4e158264aeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:44.318127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e646a0-0330-45a5-a265-70378997b556 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:44.320813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e646a0-0330-45a5-a265-70378997b556 service glance] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:44.334417 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1234/2420] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:48:44.335320 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e646a0-0330-45a5-a265-70378997b556 service glance] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:44.354633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e646a0-0330-45a5-a265-70378997b556 service glance] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:44.365838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81d14d-cdf0-4c8e-8d1f-50aadbecd3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:44.366981 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81d14d-cdf0-4c8e-8d1f-50aadbecd3fd admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:44.367343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81d14d-cdf0-4c8e-8d1f-50aadbecd3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:44.367685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81d14d-cdf0-4c8e-8d1f-50aadbecd3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:44.371157 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81d14d-cdf0-4c8e-8d1f-50aadbecd3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:44.374251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81d14d-cdf0-4c8e-8d1f-50aadbecd3fd admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:44.399865 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1235/2421] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:44.400865 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81d14d-cdf0-4c8e-8d1f-50aadbecd3fd admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:44.401451 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1187/2422] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jul 12 13:48:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:44.415259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81d14d-cdf0-4c8e-8d1f-50aadbecd3fd admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:44.416660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f5522-9109-494f-9ea6-4e158264aeba None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:44.418889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-583f5522-9109-494f-9ea6-4e158264aeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:44.425152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfbc71-0fe1-4a44-a84d-c11e37e803e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:44.428386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfbc71-0fe1-4a44-a84d-c11e37e803e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:44.428686 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfbc71-0fe1-4a44-a84d-c11e37e803e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:44.428977 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfbc71-0fe1-4a44-a84d-c11e37e803e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:44.433448 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfbc71-0fe1-4a44-a84d-c11e37e803e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:44.434647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfbc71-0fe1-4a44-a84d-c11e37e803e3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:44.455920 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:44.468285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1236/2423] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:48:44.470493 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfbc71-0fe1-4a44-a84d-c11e37e803e3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:44.508357 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfbc71-0fe1-4a44-a84d-c11e37e803e3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:44.524302 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28057ca4-f7d6-49e1-b872-e365a1427f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:44.525821 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28057ca4-f7d6-49e1-b872-e365a1427f4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:44.526419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28057ca4-f7d6-49e1-b872-e365a1427f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:44.526645 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28057ca4-f7d6-49e1-b872-e365a1427f4f admin admin] PATH_INFO: `/v3/projects/dc981a893df54f079e63399ace23ef1c/users/3dd90f756d724016b60c80e9c78cce41/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:44.538240 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f5522-9109-494f-9ea6-4e158264aeba None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:44.542791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:44.550432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0cf122-6dfb-4786-81ae-d7d8d34f007e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:44.551201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cf122-6dfb-4786-81ae-d7d8d34f007e service swift] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:44.551584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cf122-6dfb-4786-81ae-d7d8d34f007e service swift] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:44.551962 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cf122-6dfb-4786-81ae-d7d8d34f007e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:44.566947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0cf122-6dfb-4786-81ae-d7d8d34f007e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:44.568741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0cf122-6dfb-4786-81ae-d7d8d34f007e service swift] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:44.570637 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28057ca4-f7d6-49e1-b872-e365a1427f4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc981a893df54f079e63399ace23ef1c, user_id=3dd90f756d724016b60c80e9c78cce41, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:44.575653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28057ca4-f7d6-49e1-b872-e365a1427f4f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:44.599585 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1188/2424] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:48:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:48:44.601217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1237/2425] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:48:44 2022] PUT /identity/v3/projects/dc981a893df54f079e63399ace23ef1c/users/3dd90f756d724016b60c80e9c78cce41/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:48:44.602392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c853a4-0bee-4549-87f8-23ac53929043 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:44.603002 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c853a4-0bee-4549-87f8-23ac53929043 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:44.604019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c853a4-0bee-4549-87f8-23ac53929043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:44.607234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3b76f-b785-43f9-a787-b67ea65dd5d2 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:44.607525 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3b76f-b785-43f9-a787-b67ea65dd5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:44.607803 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3b76f-b785-43f9-a787-b67ea65dd5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:44.632262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7ca3b76f-b785-43f9-a787-b67ea65dd5d2 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:44.633556 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7ca3b76f-b785-43f9-a787-b67ea65dd5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:44.699345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-01c853a4-0bee-4549-87f8-23ac53929043 None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:48:44.711786 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-01c853a4-0bee-4549-87f8-23ac53929043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:44.833781 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1238/2426] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:44.846855 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3b76f-b785-43f9-a787-b67ea65dd5d2 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:44.848936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7ca3b76f-b785-43f9-a787-b67ea65dd5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:44.856633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1189/2427] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jul 12 13:48:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:48:44.906683 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c853a4-0bee-4549-87f8-23ac53929043 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:45.011780 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3b76f-b785-43f9-a787-b67ea65dd5d2 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:45.021408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17795c95-c54c-4812-9444-fae1b122392b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:45.022040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17795c95-c54c-4812-9444-fae1b122392b service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:45.022327 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17795c95-c54c-4812-9444-fae1b122392b service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:45.023482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17795c95-c54c-4812-9444-fae1b122392b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:45.045571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17795c95-c54c-4812-9444-fae1b122392b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:45.048110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17795c95-c54c-4812-9444-fae1b122392b service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:45.060650 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c853a4-0bee-4549-87f8-23ac53929043 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:45.071360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5f6cf-4b53-4cd7-9a94-8b4d76375756 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:45.074356 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5f6cf-4b53-4cd7-9a94-8b4d76375756 service swift] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:45.076051 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5f6cf-4b53-4cd7-9a94-8b4d76375756 service swift] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:45.076187 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5f6cf-4b53-4cd7-9a94-8b4d76375756 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:45.082996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1239/2428] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:48:45.093529 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5f6cf-4b53-4cd7-9a94-8b4d76375756 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:45.095732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5f6cf-4b53-4cd7-9a94-8b4d76375756 service swift] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:45.113979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1190/2429] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:48:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 13:48:49.831552 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17795c95-c54c-4812-9444-fae1b122392b service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:49.856578 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17795c95-c54c-4812-9444-fae1b122392b service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:49.884714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd91e4c-a102-4f0b-a7a1-4cf8611ff7ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:49.886519 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd91e4c-a102-4f0b-a7a1-4cf8611ff7ca service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:49.887342 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd91e4c-a102-4f0b-a7a1-4cf8611ff7ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:49.889436 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd91e4c-a102-4f0b-a7a1-4cf8611ff7ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:49.894838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ddd91e4c-a102-4f0b-a7a1-4cf8611ff7ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:49.959087 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd91e4c-a102-4f0b-a7a1-4cf8611ff7ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:49.964319 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd91e4c-a102-4f0b-a7a1-4cf8611ff7ca service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:50.012311 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1240/2430] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:48:55.106179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=80 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:48:59.692699 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5f6cf-4b53-4cd7-9a94-8b4d76375756 service swift] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:59.696503 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e0b5f6cf-4b53-4cd7-9a94-8b4d76375756 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:59.869651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd91e4c-a102-4f0b-a7a1-4cf8611ff7ca service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:48:59.871368 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ddd91e4c-a102-4f0b-a7a1-4cf8611ff7ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:48:59.895985 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd91e4c-a102-4f0b-a7a1-4cf8611ff7ca service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:59.912467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666349cf-f063-4d4d-8cdc-01b44b7b5bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:59.913477 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666349cf-f063-4d4d-8cdc-01b44b7b5bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:59.913926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666349cf-f063-4d4d-8cdc-01b44b7b5bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:59.914303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666349cf-f063-4d4d-8cdc-01b44b7b5bcb admin admin] PATH_INFO: `/v3/users/2ad8ebe54a1d42eda9b23108f92a1d23` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:59.920578 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666349cf-f063-4d4d-8cdc-01b44b7b5bcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ad8ebe54a1d42eda9b23108f92a1d23)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:48:59.922877 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666349cf-f063-4d4d-8cdc-01b44b7b5bcb admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:48:59.943171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5f6cf-4b53-4cd7-9a94-8b4d76375756 service swift] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:48:59.955055 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff38c88-650e-4e24-a8d0-1dda1ab261b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1308deea1ea4629a1ac22d4b903ad4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:48:59.955718 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff38c88-650e-4e24-a8d0-1dda1ab261b3 service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:48:59.956059 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff38c88-650e-4e24-a8d0-1dda1ab261b3 service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:48:59.956342 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff38c88-650e-4e24-a8d0-1dda1ab261b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:48:59.961607 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0ff38c88-650e-4e24-a8d0-1dda1ab261b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:00.016152 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff38c88-650e-4e24-a8d0-1dda1ab261b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:00.018086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff38c88-650e-4e24-a8d0-1dda1ab261b3 service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:00.039659 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-666349cf-f063-4d4d-8cdc-01b44b7b5bcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ad8ebe54a1d42eda9b23108f92a1d23'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:00.042086 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-666349cf-f063-4d4d-8cdc-01b44b7b5bcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ad8ebe54a1d42eda9b23108f92a1d23'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:00.045476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-666349cf-f063-4d4d-8cdc-01b44b7b5bcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ad8ebe54a1d42eda9b23108f92a1d23'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:00.071484 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1191/2431] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:48:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:00.094905 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-666349cf-f063-4d4d-8cdc-01b44b7b5bcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ad8ebe54a1d42eda9b23108f92a1d23'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:00.105501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:00.109525 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-666349cf-f063-4d4d-8cdc-01b44b7b5bcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ad8ebe54a1d42eda9b23108f92a1d23'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:00.166354 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1241/2432] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:48:59 2022] DELETE /identity/v3/users/2ad8ebe54a1d42eda9b23108f92a1d23 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:00.272272 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f921b39-4abd-4522-8066-a9cb572de31e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:00.272622 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f921b39-4abd-4522-8066-a9cb572de31e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:00.272870 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f921b39-4abd-4522-8066-a9cb572de31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:00.390302 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4f921b39-4abd-4522-8066-a9cb572de31e None None] MFA Rules not processed for user `3dd90f756d724016b60c80e9c78cce41`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:00.393888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4f921b39-4abd-4522-8066-a9cb572de31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:00.687235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1192/2433] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:00.688529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666349cf-f063-4d4d-8cdc-01b44b7b5bcb admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:00.690119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-666349cf-f063-4d4d-8cdc-01b44b7b5bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:00.745525 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f921b39-4abd-4522-8066-a9cb572de31e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:00.747282 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4f921b39-4abd-4522-8066-a9cb572de31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:00.860161 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f921b39-4abd-4522-8066-a9cb572de31e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:00.879138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189e82f-e008-466d-9996-48d5c0efe5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:00.882169 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e82f-e008-466d-9996-48d5c0efe5a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:00.882476 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e82f-e008-466d-9996-48d5c0efe5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:00.882766 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e82f-e008-466d-9996-48d5c0efe5a1 admin admin] PATH_INFO: `/v3/projects/5f431e3ab50a4133916f6175a2ef1b4f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:00.885093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189e82f-e008-466d-9996-48d5c0efe5a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f431e3ab50a4133916f6175a2ef1b4f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:00.893007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189e82f-e008-466d-9996-48d5c0efe5a1 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:00.901031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666349cf-f063-4d4d-8cdc-01b44b7b5bcb admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:00.909908 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea321cfa-822b-437c-8784-807db1a8d555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:00.912035 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea321cfa-822b-437c-8784-807db1a8d555 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:00.914961 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea321cfa-822b-437c-8784-807db1a8d555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:00.916199 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea321cfa-822b-437c-8784-807db1a8d555 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:00.917314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea321cfa-822b-437c-8784-807db1a8d555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:00.922432 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea321cfa-822b-437c-8784-807db1a8d555 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:00.976577 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1242/2434] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:00 2022] POST /identity/v3/projects => generated 373 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:00.980790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:00.984055 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea321cfa-822b-437c-8784-807db1a8d555 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:01.041688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-5189e82f-e008-466d-9996-48d5c0efe5a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f431e3ab50a4133916f6175a2ef1b4f', 'request_id': 'req-5189e82f-e008-466d-9996-48d5c0efe5a1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:01.058253 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-5189e82f-e008-466d-9996-48d5c0efe5a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f431e3ab50a4133916f6175a2ef1b4f', 'request_id': 'req-5189e82f-e008-466d-9996-48d5c0efe5a1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:01.081056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-5189e82f-e008-466d-9996-48d5c0efe5a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f431e3ab50a4133916f6175a2ef1b4f', 'request_id': 'req-5189e82f-e008-466d-9996-48d5c0efe5a1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:01.123504 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-5189e82f-e008-466d-9996-48d5c0efe5a1 admin admin] The token cache is being invalidate because project 5f431e3ab50a4133916f6175a2ef1b4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:01.124053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-5189e82f-e008-466d-9996-48d5c0efe5a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:01.127346 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-5189e82f-e008-466d-9996-48d5c0efe5a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:01.130638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1193/2435] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:00 2022] DELETE /identity/v3/projects/5f431e3ab50a4133916f6175a2ef1b4f => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:01.135473 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bb827-8712-4846-858e-a2f5bc1bf1c4 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:01.136139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bb827-8712-4846-858e-a2f5bc1bf1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:01.136581 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bb827-8712-4846-858e-a2f5bc1bf1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:01.158219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea321cfa-822b-437c-8784-807db1a8d555 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:01.183027 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-624bb827-8712-4846-858e-a2f5bc1bf1c4 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:01.184587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-624bb827-8712-4846-858e-a2f5bc1bf1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:01.336606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d91db-4adf-435f-8981-ab3ea33db852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:01.337483 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d91db-4adf-435f-8981-ab3ea33db852 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:01.337875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d91db-4adf-435f-8981-ab3ea33db852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:01.338267 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d91db-4adf-435f-8981-ab3ea33db852 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:01.340206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d91db-4adf-435f-8981-ab3ea33db852 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:01.340422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1194/2436] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:01.342607 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d91db-4adf-435f-8981-ab3ea33db852 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:01.348393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96645711-7cf7-41a5-bf2d-93f9028f3c29 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:01.349455 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96645711-7cf7-41a5-bf2d-93f9028f3c29 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:01.349871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96645711-7cf7-41a5-bf2d-93f9028f3c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:01.391476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1243/2437] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:00 2022] POST /identity/v3/users => generated 293 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:01.392738 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d91db-4adf-435f-8981-ab3ea33db852 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:01.395987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3f5d91db-4adf-435f-8981-ab3ea33db852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:01.397075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-96645711-7cf7-41a5-bf2d-93f9028f3c29 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:01.399765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-96645711-7cf7-41a5-bf2d-93f9028f3c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:01.425190 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1195/2438] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:49:01.428153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96645711-7cf7-41a5-bf2d-93f9028f3c29 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:01.438690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d91db-4adf-435f-8981-ab3ea33db852 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:01.442386 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96645711-7cf7-41a5-bf2d-93f9028f3c29 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:01.456590 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6bf91-de59-4796-892a-f21d63d20620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:01.458422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6bf91-de59-4796-892a-f21d63d20620 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:01.458923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6bf91-de59-4796-892a-f21d63d20620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:01.459445 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6bf91-de59-4796-892a-f21d63d20620 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:01.460439 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe171e67-4bcf-4d6c-9ce0-4bff651dc556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:01.464188 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe171e67-4bcf-4d6c-9ce0-4bff651dc556 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:01.464567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe171e67-4bcf-4d6c-9ce0-4bff651dc556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:01.465117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe171e67-4bcf-4d6c-9ce0-4bff651dc556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:01.465906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe171e67-4bcf-4d6c-9ce0-4bff651dc556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:01.471174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe171e67-4bcf-4d6c-9ce0-4bff651dc556 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:01.471886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6bf91-de59-4796-892a-f21d63d20620 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:49:01.472442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6bf91-de59-4796-892a-f21d63d20620 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:01.474331 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6bf91-de59-4796-892a-f21d63d20620 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:01.479421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:01.496164 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1196/2439] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:49:01.498279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6bf91-de59-4796-892a-f21d63d20620 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:01.502907 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1244/2440] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:01.504084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe171e67-4bcf-4d6c-9ce0-4bff651dc556 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:01.521357 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe171e67-4bcf-4d6c-9ce0-4bff651dc556 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:01.531202 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6bf91-de59-4796-892a-f21d63d20620 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:01.535731 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec03429-6dc3-41b0-9469-1cf43ae8c6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:01.540256 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec03429-6dc3-41b0-9469-1cf43ae8c6f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:01.540801 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec03429-6dc3-41b0-9469-1cf43ae8c6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:01.541046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec03429-6dc3-41b0-9469-1cf43ae8c6f4 admin admin] PATH_INFO: `/v3/projects/5b8c618fc5664694ad0bb1f5fb414ec1/users/9fb6b21121b947658272e0f195a79f01/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:01.546194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e283c-9ce1-4a90-98a8-ce7b782f3b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:01.550654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e283c-9ce1-4a90-98a8-ce7b782f3b99 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:01.551262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e283c-9ce1-4a90-98a8-ce7b782f3b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:01.551758 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e283c-9ce1-4a90-98a8-ce7b782f3b99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:01.552690 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e283c-9ce1-4a90-98a8-ce7b782f3b99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:01.559579 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e283c-9ce1-4a90-98a8-ce7b782f3b99 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:01.578855 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec03429-6dc3-41b0-9469-1cf43ae8c6f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b8c618fc5664694ad0bb1f5fb414ec1, user_id=9fb6b21121b947658272e0f195a79f01, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:01.583791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec03429-6dc3-41b0-9469-1cf43ae8c6f4 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:01.615903 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1245/2441] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:01 2022] PUT /identity/v3/projects/5b8c618fc5664694ad0bb1f5fb414ec1/users/9fb6b21121b947658272e0f195a79f01/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:01.619770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec03429-6dc3-41b0-9469-1cf43ae8c6f4 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:01.628204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1197/2442] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:01 2022] POST /identity/v3/projects => generated 367 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:01.629672 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e283c-9ce1-4a90-98a8-ce7b782f3b99 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:01.756637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e283c-9ce1-4a90-98a8-ce7b782f3b99 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:01.762963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f75bc1-1cdf-49b6-b734-96d0166b096a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:01.764127 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f75bc1-1cdf-49b6-b734-96d0166b096a admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:01.764539 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f75bc1-1cdf-49b6-b734-96d0166b096a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:01.764909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f75bc1-1cdf-49b6-b734-96d0166b096a admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:01.765896 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f75bc1-1cdf-49b6-b734-96d0166b096a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:01.768925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f75bc1-1cdf-49b6-b734-96d0166b096a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:01.781095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec03429-6dc3-41b0-9469-1cf43ae8c6f4 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:01.799961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:01.823024 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69de50-f050-4a06-8971-8a2fcf2b3d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:01.831487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69de50-f050-4a06-8971-8a2fcf2b3d51 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:01.831487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69de50-f050-4a06-8971-8a2fcf2b3d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:01.831487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69de50-f050-4a06-8971-8a2fcf2b3d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:01.831487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69de50-f050-4a06-8971-8a2fcf2b3d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:01.834170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69de50-f050-4a06-8971-8a2fcf2b3d51 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:01.834846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1198/2443] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:01 2022] POST /identity/v3/users => generated 284 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:01.835878 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f75bc1-1cdf-49b6-b734-96d0166b096a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:01.871929 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f75bc1-1cdf-49b6-b734-96d0166b096a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:01.876119 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1246/2444] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:49:01.885381 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69de50-f050-4a06-8971-8a2fcf2b3d51 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:01.889793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec1cde-6697-4d51-b460-90793ff89a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:01.890504 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec1cde-6697-4d51-b460-90793ff89a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:01.890788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec1cde-6697-4d51-b460-90793ff89a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:01.891060 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec1cde-6697-4d51-b460-90793ff89a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:01.891800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec1cde-6697-4d51-b460-90793ff89a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:01.904105 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec1cde-6697-4d51-b460-90793ff89a3f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:01.921018 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69de50-f050-4a06-8971-8a2fcf2b3d51 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:01.939151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368820dd-086f-4e74-9e7f-58a726f71f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:01.940031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368820dd-086f-4e74-9e7f-58a726f71f24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:01.940312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368820dd-086f-4e74-9e7f-58a726f71f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:01.947588 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368820dd-086f-4e74-9e7f-58a726f71f24 admin admin] PATH_INFO: `/v3/projects/5b8c618fc5664694ad0bb1f5fb414ec1/users/9fb6b21121b947658272e0f195a79f01/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:01.947799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1199/2445] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:01.949245 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec1cde-6697-4d51-b460-90793ff89a3f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:01.960149 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368820dd-086f-4e74-9e7f-58a726f71f24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b8c618fc5664694ad0bb1f5fb414ec1, user_id=9fb6b21121b947658272e0f195a79f01, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:01.969416 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368820dd-086f-4e74-9e7f-58a726f71f24 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:02.004042 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec1cde-6697-4d51-b460-90793ff89a3f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:02.021323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae393b4-50a8-4bb0-8996-d8a36e9cedc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:02.021323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae393b4-50a8-4bb0-8996-d8a36e9cedc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:02.021323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae393b4-50a8-4bb0-8996-d8a36e9cedc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:02.021695 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae393b4-50a8-4bb0-8996-d8a36e9cedc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:02.022247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1247/2446] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:01 2022] PUT /identity/v3/projects/5b8c618fc5664694ad0bb1f5fb414ec1/users/9fb6b21121b947658272e0f195a79f01/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:02.022426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae393b4-50a8-4bb0-8996-d8a36e9cedc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:02.041825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae393b4-50a8-4bb0-8996-d8a36e9cedc5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:02.077234 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1200/2447] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:02.081478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368820dd-086f-4e74-9e7f-58a726f71f24 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:02.290349 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368820dd-086f-4e74-9e7f-58a726f71f24 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:02.311133 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5c02b-5489-4976-a574-5a76a21aae19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:02.312135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5c02b-5489-4976-a574-5a76a21aae19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:02.312377 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5c02b-5489-4976-a574-5a76a21aae19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:02.312639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5c02b-5489-4976-a574-5a76a21aae19 admin admin] PATH_INFO: `/v3/projects/52261f200bb7404b8234d60957ebe2cc/users/937c588edf3b48d68e22da195bdef018/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:02.351263 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5c02b-5489-4976-a574-5a76a21aae19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52261f200bb7404b8234d60957ebe2cc, user_id=937c588edf3b48d68e22da195bdef018, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:02.356268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5c02b-5489-4976-a574-5a76a21aae19 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:02.402646 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1248/2448] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:02 2022] PUT /identity/v3/projects/52261f200bb7404b8234d60957ebe2cc/users/937c588edf3b48d68e22da195bdef018/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:02.406286 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade50b02-8719-4f81-b91e-3e4c836dc2a7 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:02.406788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade50b02-8719-4f81-b91e-3e4c836dc2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:02.407231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade50b02-8719-4f81-b91e-3e4c836dc2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:02.469263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ade50b02-8719-4f81-b91e-3e4c836dc2a7 None None] MFA Rules not processed for user `937c588edf3b48d68e22da195bdef018`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:02.471910 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ade50b02-8719-4f81-b91e-3e4c836dc2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:02.642035 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1201/2449] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:02.654523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5c02b-5489-4976-a574-5a76a21aae19 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:02.656382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-88c5c02b-5489-4976-a574-5a76a21aae19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:02.753513 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:02.779236 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5c02b-5489-4976-a574-5a76a21aae19 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:02.811043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9841e0b-86d0-46a9-a37a-d2ba0ed1d587 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:02.812955 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9841e0b-86d0-46a9-a37a-d2ba0ed1d587 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:02.815436 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9841e0b-86d0-46a9-a37a-d2ba0ed1d587 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:02.821486 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9841e0b-86d0-46a9-a37a-d2ba0ed1d587 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:02.861140 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9841e0b-86d0-46a9-a37a-d2ba0ed1d587 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:02.863276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9841e0b-86d0-46a9-a37a-d2ba0ed1d587 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:02.886848 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1249/2450] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:04.517592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55b220-9f28-43f0-92de-f44414765283 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:04.518707 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55b220-9f28-43f0-92de-f44414765283 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:04.519177 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55b220-9f28-43f0-92de-f44414765283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:04.539813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0e55b220-9f28-43f0-92de-f44414765283 None None] MFA Rules not processed for user `ab68f09ccf4d4ac9901d4ed792b551c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:04.541129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0e55b220-9f28-43f0-92de-f44414765283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:04.638086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1202/2451] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:06.716178 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9841e0b-86d0-46a9-a37a-d2ba0ed1d587 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:06.718676 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b9841e0b-86d0-46a9-a37a-d2ba0ed1d587 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:06.890939 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9841e0b-86d0-46a9-a37a-d2ba0ed1d587 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:06.896546 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2625c2c-de64-4cdd-bd59-75af64085663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:06.897488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2625c2c-de64-4cdd-bd59-75af64085663 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:06.898300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2625c2c-de64-4cdd-bd59-75af64085663 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:06.900441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2625c2c-de64-4cdd-bd59-75af64085663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:06.902739 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f2625c2c-de64-4cdd-bd59-75af64085663 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:06.999857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2625c2c-de64-4cdd-bd59-75af64085663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:07.001538 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2625c2c-de64-4cdd-bd59-75af64085663 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:07.029123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1250/2452] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:08.695813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1e5ac-1798-4158-9fe5-94d90d605088 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:08.696934 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1e5ac-1798-4158-9fe5-94d90d605088 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:08.696934 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1e5ac-1798-4158-9fe5-94d90d605088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:08.735008 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ebc1e5ac-1798-4158-9fe5-94d90d605088 None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:08.736808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ebc1e5ac-1798-4158-9fe5-94d90d605088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:08.905832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1203/2453] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jul 12 13:49:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:08.920932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2625c2c-de64-4cdd-bd59-75af64085663 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:08.922981 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f2625c2c-de64-4cdd-bd59-75af64085663 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:09.079757 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2625c2c-de64-4cdd-bd59-75af64085663 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:09.085384 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d61d2b-1eea-4892-adcb-3924ba54e181 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:09.086071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d61d2b-1eea-4892-adcb-3924ba54e181 service swift] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:09.086356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d61d2b-1eea-4892-adcb-3924ba54e181 service swift] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:09.086683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d61d2b-1eea-4892-adcb-3924ba54e181 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:09.096731 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d61d2b-1eea-4892-adcb-3924ba54e181 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:09.098742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d61d2b-1eea-4892-adcb-3924ba54e181 service swift] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:09.109736 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1251/2454] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:49:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:49:11.564522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51daae-5027-4f6c-8515-43185d0fb817 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:11.564522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51daae-5027-4f6c-8515-43185d0fb817 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:11.564522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51daae-5027-4f6c-8515-43185d0fb817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:11.600213 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-2c51daae-5027-4f6c-8515-43185d0fb817 None None] MFA Rules not processed for user `9fb6b21121b947658272e0f195a79f01`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:11.601717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2c51daae-5027-4f6c-8515-43185d0fb817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:11.607610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:11.798541 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1204/2455] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:11.801250 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d61d2b-1eea-4892-adcb-3924ba54e181 service swift] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:11.802761 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-50d61d2b-1eea-4892-adcb-3924ba54e181 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:11.863693 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:12.028967 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d61d2b-1eea-4892-adcb-3924ba54e181 service swift] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:12.044233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293ad9c-46b3-4b2f-b464-d8f5ca4bcb26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:12.046144 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293ad9c-46b3-4b2f-b464-d8f5ca4bcb26 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:12.046449 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293ad9c-46b3-4b2f-b464-d8f5ca4bcb26 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:12.046735 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293ad9c-46b3-4b2f-b464-d8f5ca4bcb26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:12.049405 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e293ad9c-46b3-4b2f-b464-d8f5ca4bcb26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:12.188861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293ad9c-46b3-4b2f-b464-d8f5ca4bcb26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:12.191180 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293ad9c-46b3-4b2f-b464-d8f5ca4bcb26 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:12.218177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1252/2456] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:14.162664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51daae-5027-4f6c-8515-43185d0fb817 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:14.182490 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51daae-5027-4f6c-8515-43185d0fb817 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:14.197592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcd9b5-f4cc-4265-a0ab-0f92c7cf0c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:14.199253 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcd9b5-f4cc-4265-a0ab-0f92c7cf0c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:14.199828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcd9b5-f4cc-4265-a0ab-0f92c7cf0c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:14.200345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcd9b5-f4cc-4265-a0ab-0f92c7cf0c5e admin admin] PATH_INFO: `/v3/users/937c588edf3b48d68e22da195bdef018` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:14.202320 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcd9b5-f4cc-4265-a0ab-0f92c7cf0c5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=937c588edf3b48d68e22da195bdef018)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:14.205078 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcd9b5-f4cc-4265-a0ab-0f92c7cf0c5e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:14.428010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c3dcd9b5-f4cc-4265-a0ab-0f92c7cf0c5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '937c588edf3b48d68e22da195bdef018'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:14.511923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c3dcd9b5-f4cc-4265-a0ab-0f92c7cf0c5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '937c588edf3b48d68e22da195bdef018'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:14.548087 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c3dcd9b5-f4cc-4265-a0ab-0f92c7cf0c5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '937c588edf3b48d68e22da195bdef018'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:14.553312 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c3dcd9b5-f4cc-4265-a0ab-0f92c7cf0c5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '937c588edf3b48d68e22da195bdef018'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:14.597390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c3dcd9b5-f4cc-4265-a0ab-0f92c7cf0c5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '937c588edf3b48d68e22da195bdef018'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:14.634403 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1205/2457] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:14 2022] DELETE /identity/v3/users/937c588edf3b48d68e22da195bdef018 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:14.638793 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a06a4a-d785-4832-9414-6009bed21e09 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:14.639358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a06a4a-d785-4832-9414-6009bed21e09 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:14.639358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a06a4a-d785-4832-9414-6009bed21e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:14.687405 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-15a06a4a-d785-4832-9414-6009bed21e09 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:14.689085 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-15a06a4a-d785-4832-9414-6009bed21e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:14.801548 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1253/2458] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:14.824949 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcd9b5-f4cc-4265-a0ab-0f92c7cf0c5e admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:14.827811 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c3dcd9b5-f4cc-4265-a0ab-0f92c7cf0c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:14.940468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcd9b5-f4cc-4265-a0ab-0f92c7cf0c5e admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:14.944763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dca1cf-23b4-4bb0-9367-10a13c4cb838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:14.945554 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dca1cf-23b4-4bb0-9367-10a13c4cb838 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:14.945914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dca1cf-23b4-4bb0-9367-10a13c4cb838 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:14.946260 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dca1cf-23b4-4bb0-9367-10a13c4cb838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:14.974433 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dca1cf-23b4-4bb0-9367-10a13c4cb838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:14.976508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dca1cf-23b4-4bb0-9367-10a13c4cb838 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:14.997729 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1206/2459] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:17.033074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d3836-f10b-4b23-8b59-bf7035a7b349 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:17.033610 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d3836-f10b-4b23-8b59-bf7035a7b349 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:17.033702 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d3836-f10b-4b23-8b59-bf7035a7b349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:17.087876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-096d3836-f10b-4b23-8b59-bf7035a7b349 None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:17.090264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59d013-0cfe-48fd-8588-4d7c27078a2f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:17.091659 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59d013-0cfe-48fd-8588-4d7c27078a2f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:17.092621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59d013-0cfe-48fd-8588-4d7c27078a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:17.093774 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-096d3836-f10b-4b23-8b59-bf7035a7b349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:17.127884 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0a59d013-0cfe-48fd-8588-4d7c27078a2f None None] MFA Rules not processed for user `3dd90f756d724016b60c80e9c78cce41`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:17.129987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0a59d013-0cfe-48fd-8588-4d7c27078a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:17.151085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:17.325325 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1254/2460] 10.222.0.52 () {58 vars in 967 bytes} [Tue Jul 12 13:49:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:17.359114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1207/2461] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:17.362457 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d3836-f10b-4b23-8b59-bf7035a7b349 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:17.364448 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-096d3836-f10b-4b23-8b59-bf7035a7b349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:17.391811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:17.405697 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d3836-f10b-4b23-8b59-bf7035a7b349 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:17.405886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59d013-0cfe-48fd-8588-4d7c27078a2f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:17.410611 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0a59d013-0cfe-48fd-8588-4d7c27078a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:17.417984 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9de132-d79b-4e93-a08b-bd91a5e39fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:17.418911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9de132-d79b-4e93-a08b-bd91a5e39fba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:17.419286 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9de132-d79b-4e93-a08b-bd91a5e39fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:17.419659 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9de132-d79b-4e93-a08b-bd91a5e39fba admin admin] PATH_INFO: `/v3/projects/52261f200bb7404b8234d60957ebe2cc` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:17.422844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9de132-d79b-4e93-a08b-bd91a5e39fba admin admin] RBAC: Authorizing `identity:delete_project(project_id=52261f200bb7404b8234d60957ebe2cc)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:17.426706 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9de132-d79b-4e93-a08b-bd91a5e39fba admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:17.544364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ec9de132-d79b-4e93-a08b-bd91a5e39fba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52261f200bb7404b8234d60957ebe2cc', 'request_id': 'req-ec9de132-d79b-4e93-a08b-bd91a5e39fba'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:17.558983 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ec9de132-d79b-4e93-a08b-bd91a5e39fba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52261f200bb7404b8234d60957ebe2cc', 'request_id': 'req-ec9de132-d79b-4e93-a08b-bd91a5e39fba'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:17.575608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59d013-0cfe-48fd-8588-4d7c27078a2f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:17.577589 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ec9de132-d79b-4e93-a08b-bd91a5e39fba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52261f200bb7404b8234d60957ebe2cc', 'request_id': 'req-ec9de132-d79b-4e93-a08b-bd91a5e39fba'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:17.604695 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ec9de132-d79b-4e93-a08b-bd91a5e39fba admin admin] The token cache is being invalidate because project 52261f200bb7404b8234d60957ebe2cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:17.607781 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ec9de132-d79b-4e93-a08b-bd91a5e39fba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:17.624007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ec9de132-d79b-4e93-a08b-bd91a5e39fba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:17.628913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1255/2462] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:17 2022] DELETE /identity/v3/projects/52261f200bb7404b8234d60957ebe2cc => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:17.633900 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984b253-2236-4cda-ab08-deeb7f870f17 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:17.634239 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984b253-2236-4cda-ab08-deeb7f870f17 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:17.634481 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984b253-2236-4cda-ab08-deeb7f870f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:17.698055 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d984b253-2236-4cda-ab08-deeb7f870f17 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:17.700337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d984b253-2236-4cda-ab08-deeb7f870f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:17.770390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95af2a-9d10-4ac9-b4d2-5f7ab94c43ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:17.773518 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95af2a-9d10-4ac9-b4d2-5f7ab94c43ea service swift] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:17.778223 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95af2a-9d10-4ac9-b4d2-5f7ab94c43ea service swift] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:17.778790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95af2a-9d10-4ac9-b4d2-5f7ab94c43ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:17.797855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2c95af2a-9d10-4ac9-b4d2-5f7ab94c43ea service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:17.900007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1256/2463] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:17.901754 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91959fba-8c88-4620-9476-238f18eb1762 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:17.902103 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91959fba-8c88-4620-9476-238f18eb1762 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:17.902364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91959fba-8c88-4620-9476-238f18eb1762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:17.963743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-91959fba-8c88-4620-9476-238f18eb1762 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:17.965012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-91959fba-8c88-4620-9476-238f18eb1762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:17.972433 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95af2a-9d10-4ac9-b4d2-5f7ab94c43ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:17.974121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95af2a-9d10-4ac9-b4d2-5f7ab94c43ea service swift] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:18.004330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1208/2464] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:49:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:49:18.007582 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1257/2465] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:18.010806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95af2a-9d10-4ac9-b4d2-5f7ab94c43ea service swift] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:18.034376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:18.063361 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95af2a-9d10-4ac9-b4d2-5f7ab94c43ea service swift] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:18.076011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37650930-70b9-4f7b-ae3a-0729a32e3ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:18.076947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37650930-70b9-4f7b-ae3a-0729a32e3ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:18.077231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37650930-70b9-4f7b-ae3a-0729a32e3ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:18.077505 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37650930-70b9-4f7b-ae3a-0729a32e3ea6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:18.078278 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37650930-70b9-4f7b-ae3a-0729a32e3ea6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:49:18.078580 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37650930-70b9-4f7b-ae3a-0729a32e3ea6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:18.080195 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37650930-70b9-4f7b-ae3a-0729a32e3ea6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:18.110653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1209/2466] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:49:18.114714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91959fba-8c88-4620-9476-238f18eb1762 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:18.129428 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91959fba-8c88-4620-9476-238f18eb1762 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:18.142325 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a5bef-99bc-4c59-a09e-9796e1ce8572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:18.143091 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a5bef-99bc-4c59-a09e-9796e1ce8572 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:18.143378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a5bef-99bc-4c59-a09e-9796e1ce8572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:18.143643 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a5bef-99bc-4c59-a09e-9796e1ce8572 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:18.144352 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a5bef-99bc-4c59-a09e-9796e1ce8572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:18.147120 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a5bef-99bc-4c59-a09e-9796e1ce8572 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:18.225059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1258/2467] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:18 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:49:18.230751 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37650930-70b9-4f7b-ae3a-0729a32e3ea6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:18.249339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37650930-70b9-4f7b-ae3a-0729a32e3ea6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:18.428292 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b0278-5ebf-411d-a2a9-fe4d211cc2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:18.435263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b0278-5ebf-411d-a2a9-fe4d211cc2ee admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:18.435587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b0278-5ebf-411d-a2a9-fe4d211cc2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:18.436084 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b0278-5ebf-411d-a2a9-fe4d211cc2ee admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:18.436903 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b0278-5ebf-411d-a2a9-fe4d211cc2ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:18.439692 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b0278-5ebf-411d-a2a9-fe4d211cc2ee admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:18.510153 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a5bef-99bc-4c59-a09e-9796e1ce8572 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:18.510788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1210/2468] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:18 2022] POST /identity/v3/users => generated 283 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:18.523812 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a5bef-99bc-4c59-a09e-9796e1ce8572 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:18.532192 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ab0a9-d50f-4ecb-861c-5609e9fe4cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:18.533010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ab0a9-d50f-4ecb-861c-5609e9fe4cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:18.533433 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ab0a9-d50f-4ecb-861c-5609e9fe4cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:18.533842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ab0a9-d50f-4ecb-861c-5609e9fe4cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:18.534682 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ab0a9-d50f-4ecb-861c-5609e9fe4cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:18.537859 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ab0a9-d50f-4ecb-861c-5609e9fe4cc5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:18.563529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1259/2469] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:49:18.566986 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b0278-5ebf-411d-a2a9-fe4d211cc2ee admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:18.585590 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b0278-5ebf-411d-a2a9-fe4d211cc2ee admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:18.591619 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a8b99-2f97-4eba-bd4f-f3fa325a9bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:18.592421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a8b99-2f97-4eba-bd4f-f3fa325a9bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:18.592815 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a8b99-2f97-4eba-bd4f-f3fa325a9bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:18.593136 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a8b99-2f97-4eba-bd4f-f3fa325a9bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:18.593878 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a8b99-2f97-4eba-bd4f-f3fa325a9bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:18.595771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a8b99-2f97-4eba-bd4f-f3fa325a9bc7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:18.615804 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ab0a9-d50f-4ecb-861c-5609e9fe4cc5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:18.621799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1211/2470] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:18.622255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-627ab0a9-d50f-4ecb-861c-5609e9fe4cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:18.625038 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a8b99-2f97-4eba-bd4f-f3fa325a9bc7 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:18.648608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ab0a9-d50f-4ecb-861c-5609e9fe4cc5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:18.658649 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a8b99-2f97-4eba-bd4f-f3fa325a9bc7 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:18.661053 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c70b7-8fcd-4e42-bc70-5e0b58d7da7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:18.662468 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c70b7-8fcd-4e42-bc70-5e0b58d7da7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:18.663534 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c70b7-8fcd-4e42-bc70-5e0b58d7da7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:18.664261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c70b7-8fcd-4e42-bc70-5e0b58d7da7d admin admin] PATH_INFO: `/v3/users/a892b2511463404c818c63e870a23389` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:18.669184 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c70b7-8fcd-4e42-bc70-5e0b58d7da7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a892b2511463404c818c63e870a23389)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:18.671788 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c70b7-8fcd-4e42-bc70-5e0b58d7da7d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:18.680234 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c92339-ffc6-4899-9025-7d60f954dc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:18.682825 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c92339-ffc6-4899-9025-7d60f954dc7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:18.683222 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c92339-ffc6-4899-9025-7d60f954dc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:18.683518 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c92339-ffc6-4899-9025-7d60f954dc7d admin admin] PATH_INFO: `/v3/projects/5f640a3e00c6443297ae50ea3dfe2240/users/c3af833b4cbb407e8530a1eee20237eb/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:18.708569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c92339-ffc6-4899-9025-7d60f954dc7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f640a3e00c6443297ae50ea3dfe2240, user_id=c3af833b4cbb407e8530a1eee20237eb, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:18.714103 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c92339-ffc6-4899-9025-7d60f954dc7d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:18.746042 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1212/2471] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:18 2022] PUT /identity/v3/projects/5f640a3e00c6443297ae50ea3dfe2240/users/c3af833b4cbb407e8530a1eee20237eb/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:18.748292 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633a075-226c-4bc1-a58b-1c02a1a5a475 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:18.748581 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633a075-226c-4bc1-a58b-1c02a1a5a475 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:18.748865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633a075-226c-4bc1-a58b-1c02a1a5a475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:18.800412 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:18.810759 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e53c70b7-8fcd-4e42-bc70-5e0b58d7da7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a892b2511463404c818c63e870a23389'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:18.813111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e53c70b7-8fcd-4e42-bc70-5e0b58d7da7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a892b2511463404c818c63e870a23389'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:18.813814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e53c70b7-8fcd-4e42-bc70-5e0b58d7da7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a892b2511463404c818c63e870a23389'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:18.816504 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f633a075-226c-4bc1-a58b-1c02a1a5a475 None None] MFA Rules not processed for user `c3af833b4cbb407e8530a1eee20237eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:18.817911 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f633a075-226c-4bc1-a58b-1c02a1a5a475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:18.854374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e53c70b7-8fcd-4e42-bc70-5e0b58d7da7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a892b2511463404c818c63e870a23389'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:18.875419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e53c70b7-8fcd-4e42-bc70-5e0b58d7da7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a892b2511463404c818c63e870a23389'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:18.911625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1213/2473] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:18.916230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1260/2473] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:18 2022] DELETE /identity/v3/users/a892b2511463404c818c63e870a23389 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:18.917991 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633a075-226c-4bc1-a58b-1c02a1a5a475 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:18.922609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f633a075-226c-4bc1-a58b-1c02a1a5a475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:19.027686 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:19.127372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633a075-226c-4bc1-a58b-1c02a1a5a475 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:19.255191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c70b7-8fcd-4e42-bc70-5e0b58d7da7d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:19.258067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e53c70b7-8fcd-4e42-bc70-5e0b58d7da7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:19.260996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c21168-903f-42d3-966c-b07a056fba15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:19.261893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c21168-903f-42d3-966c-b07a056fba15 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:19.262460 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c21168-903f-42d3-966c-b07a056fba15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:19.262768 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c21168-903f-42d3-966c-b07a056fba15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:19.263737 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c21168-903f-42d3-966c-b07a056fba15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:19.268651 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c21168-903f-42d3-966c-b07a056fba15 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:19.299609 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c70b7-8fcd-4e42-bc70-5e0b58d7da7d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:19.320781 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d49c5a-b373-48e9-91e8-28859e800006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:19.324501 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d49c5a-b373-48e9-91e8-28859e800006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:19.324791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d49c5a-b373-48e9-91e8-28859e800006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:19.325086 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d49c5a-b373-48e9-91e8-28859e800006 admin admin] PATH_INFO: `/v3/projects/c4e88457b140443093476f9b6d9bcff4` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:19.333374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d49c5a-b373-48e9-91e8-28859e800006 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4e88457b140443093476f9b6d9bcff4)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:19.341410 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d49c5a-b373-48e9-91e8-28859e800006 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:19.358312 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1214/2474] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:18 2022] POST /identity/v3/projects => generated 365 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:19.362012 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c21168-903f-42d3-966c-b07a056fba15 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:19.436604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c0d49c5a-b373-48e9-91e8-28859e800006 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4e88457b140443093476f9b6d9bcff4', 'request_id': 'req-c0d49c5a-b373-48e9-91e8-28859e800006'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:19.464192 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c0d49c5a-b373-48e9-91e8-28859e800006 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4e88457b140443093476f9b6d9bcff4', 'request_id': 'req-c0d49c5a-b373-48e9-91e8-28859e800006'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:19.483595 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c0d49c5a-b373-48e9-91e8-28859e800006 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4e88457b140443093476f9b6d9bcff4', 'request_id': 'req-c0d49c5a-b373-48e9-91e8-28859e800006'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:19.508311 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c21168-903f-42d3-966c-b07a056fba15 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:19.512962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c0d49c5a-b373-48e9-91e8-28859e800006 admin admin] The token cache is being invalidate because project c4e88457b140443093476f9b6d9bcff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:19.513308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c0d49c5a-b373-48e9-91e8-28859e800006 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:19.514375 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c0d49c5a-b373-48e9-91e8-28859e800006 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:19.516585 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1261/2475] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:19 2022] DELETE /identity/v3/projects/c4e88457b140443093476f9b6d9bcff4 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:19.523930 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1e02f-9f41-4663-a6c1-c2af760c3641 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:19.525355 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1e02f-9f41-4663-a6c1-c2af760c3641 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:19.526373 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1e02f-9f41-4663-a6c1-c2af760c3641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:19.584043 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4bb1e02f-9f41-4663-a6c1-c2af760c3641 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:19.588306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4bb1e02f-9f41-4663-a6c1-c2af760c3641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:19.626332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958ac6f-679b-41b1-8b5c-aac8ec3a0422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:19.628755 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958ac6f-679b-41b1-8b5c-aac8ec3a0422 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:19.629162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958ac6f-679b-41b1-8b5c-aac8ec3a0422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:19.629615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958ac6f-679b-41b1-8b5c-aac8ec3a0422 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:19.630510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958ac6f-679b-41b1-8b5c-aac8ec3a0422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:19.632782 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958ac6f-679b-41b1-8b5c-aac8ec3a0422 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:19.674010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1215/2476] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:19 2022] POST /identity/v3/users => generated 289 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:49:19.679275 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958ac6f-679b-41b1-8b5c-aac8ec3a0422 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:19.683761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3958ac6f-679b-41b1-8b5c-aac8ec3a0422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:19.698229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:19.704655 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958ac6f-679b-41b1-8b5c-aac8ec3a0422 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:19.713892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a48b7ed-f71d-48f3-befa-9ba9ad4a1a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:19.716127 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48b7ed-f71d-48f3-befa-9ba9ad4a1a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:19.717023 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48b7ed-f71d-48f3-befa-9ba9ad4a1a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:19.717654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48b7ed-f71d-48f3-befa-9ba9ad4a1a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:19.719496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a48b7ed-f71d-48f3-befa-9ba9ad4a1a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:19.723204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a48b7ed-f71d-48f3-befa-9ba9ad4a1a2d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:19.740280 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1216/2477] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:19.744156 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a48b7ed-f71d-48f3-befa-9ba9ad4a1a2d admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:19.764161 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a48b7ed-f71d-48f3-befa-9ba9ad4a1a2d admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:19.777119 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e1158-2244-4f42-8318-057c8f9850cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:19.779474 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e1158-2244-4f42-8318-057c8f9850cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:19.780040 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e1158-2244-4f42-8318-057c8f9850cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:19.780609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e1158-2244-4f42-8318-057c8f9850cf admin admin] PATH_INFO: `/v3/projects/13f533a2230b48ef9aabf597e04bcc31/users/5c7a6d39ae054f52b40350018ada7bd8/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:19.789052 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1262/2478] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:19.790753 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070ed93-80e4-4be7-bd1e-e523e012874d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:19.791166 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070ed93-80e4-4be7-bd1e-e523e012874d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:19.791315 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070ed93-80e4-4be7-bd1e-e523e012874d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:19.806011 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e1158-2244-4f42-8318-057c8f9850cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f533a2230b48ef9aabf597e04bcc31, user_id=5c7a6d39ae054f52b40350018ada7bd8, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:19.809572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e1158-2244-4f42-8318-057c8f9850cf admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:19.836405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1217/2479] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:19 2022] PUT /identity/v3/projects/13f533a2230b48ef9aabf597e04bcc31/users/5c7a6d39ae054f52b40350018ada7bd8/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:19.856873 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e1158-2244-4f42-8318-057c8f9850cf admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:19.863896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-6070ed93-80e4-4be7-bd1e-e523e012874d None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:19.865135 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6070ed93-80e4-4be7-bd1e-e523e012874d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:19.954105 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:19.980760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e1158-2244-4f42-8318-057c8f9850cf admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:19.987587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c233a77-21df-41bf-830d-48b7d86fe26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:19.988438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c233a77-21df-41bf-830d-48b7d86fe26a admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:19.988835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c233a77-21df-41bf-830d-48b7d86fe26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:19.989111 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c233a77-21df-41bf-830d-48b7d86fe26a admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:19.989844 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c233a77-21df-41bf-830d-48b7d86fe26a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:19.991800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c233a77-21df-41bf-830d-48b7d86fe26a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:19.997209 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1263/2480] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:19.998418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070ed93-80e4-4be7-bd1e-e523e012874d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:20.012970 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070ed93-80e4-4be7-bd1e-e523e012874d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:20.021458 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ce2e9-576c-482e-bb3e-7c96f047d26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:20.022182 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ce2e9-576c-482e-bb3e-7c96f047d26e admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:20.022459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ce2e9-576c-482e-bb3e-7c96f047d26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:20.022733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ce2e9-576c-482e-bb3e-7c96f047d26e admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:20.023523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ce2e9-576c-482e-bb3e-7c96f047d26e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:49:20.023837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ce2e9-576c-482e-bb3e-7c96f047d26e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:20.024028 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1218/2481] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:20.028292 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c233a77-21df-41bf-830d-48b7d86fe26a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:20.028969 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ce2e9-576c-482e-bb3e-7c96f047d26e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:20.046661 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c233a77-21df-41bf-830d-48b7d86fe26a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:20.058608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1264/2482] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:49:20.060625 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ce2e9-576c-482e-bb3e-7c96f047d26e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:20.060987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e56bcc-e35d-4793-9a7e-8e44a9c551cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:20.063031 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e56bcc-e35d-4793-9a7e-8e44a9c551cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:20.063566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e56bcc-e35d-4793-9a7e-8e44a9c551cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:20.063890 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e56bcc-e35d-4793-9a7e-8e44a9c551cd admin admin] PATH_INFO: `/v3/projects/13f533a2230b48ef9aabf597e04bcc31/users/5c7a6d39ae054f52b40350018ada7bd8/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:20.067927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e56bcc-e35d-4793-9a7e-8e44a9c551cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f533a2230b48ef9aabf597e04bcc31, user_id=5c7a6d39ae054f52b40350018ada7bd8, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:20.074177 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e56bcc-e35d-4793-9a7e-8e44a9c551cd admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:20.083477 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ce2e9-576c-482e-bb3e-7c96f047d26e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:20.090577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1219/2483] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:20 2022] PUT /identity/v3/projects/13f533a2230b48ef9aabf597e04bcc31/users/5c7a6d39ae054f52b40350018ada7bd8/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:20.104427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd36a57-471f-40f6-8ebe-70360b0ee21f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:20.105058 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd36a57-471f-40f6-8ebe-70360b0ee21f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:20.105480 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd36a57-471f-40f6-8ebe-70360b0ee21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:20.178310 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa65ca-3aba-4363-b692-50fde243299b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:20.179128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa65ca-3aba-4363-b692-50fde243299b admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:20.179489 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa65ca-3aba-4363-b692-50fde243299b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:20.179981 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa65ca-3aba-4363-b692-50fde243299b admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:20.181344 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa65ca-3aba-4363-b692-50fde243299b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:20.183436 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa65ca-3aba-4363-b692-50fde243299b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:20.190457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8bd36a57-471f-40f6-8ebe-70360b0ee21f None None] MFA Rules not processed for user `5c7a6d39ae054f52b40350018ada7bd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:20.194894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8bd36a57-471f-40f6-8ebe-70360b0ee21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:20.232306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1265/2484] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:20 2022] POST /identity/v3/projects => generated 373 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:20.236308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa65ca-3aba-4363-b692-50fde243299b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:20.327174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa65ca-3aba-4363-b692-50fde243299b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:20.334213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a5d32-b3d7-4c90-ad22-cb4f98cc9c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:20.335718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a5d32-b3d7-4c90-ad22-cb4f98cc9c64 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:20.336208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a5d32-b3d7-4c90-ad22-cb4f98cc9c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:20.336859 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a5d32-b3d7-4c90-ad22-cb4f98cc9c64 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:20.338318 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a5d32-b3d7-4c90-ad22-cb4f98cc9c64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:20.341841 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a5d32-b3d7-4c90-ad22-cb4f98cc9c64 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:20.366282 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1220/2485] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:20 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:20.380129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd36a57-471f-40f6-8ebe-70360b0ee21f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:20.382352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8bd36a57-471f-40f6-8ebe-70360b0ee21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:20.386608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1266/2486] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:20 2022] POST /identity/v3/users => generated 287 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:20.388240 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a5d32-b3d7-4c90-ad22-cb4f98cc9c64 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:20.402443 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a5d32-b3d7-4c90-ad22-cb4f98cc9c64 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:20.407696 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43458c04-1041-4af3-8504-fe618f310d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:20.408364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43458c04-1041-4af3-8504-fe618f310d9c admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:20.408647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43458c04-1041-4af3-8504-fe618f310d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:20.408922 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43458c04-1041-4af3-8504-fe618f310d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:20.409635 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43458c04-1041-4af3-8504-fe618f310d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:20.411460 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43458c04-1041-4af3-8504-fe618f310d9c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:20.426503 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1267/2487] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:49:20.428259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43458c04-1041-4af3-8504-fe618f310d9c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:20.443275 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43458c04-1041-4af3-8504-fe618f310d9c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:20.450976 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e9f3e-1b60-48d9-87fe-837b0abb2924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:20.452595 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e9f3e-1b60-48d9-87fe-837b0abb2924 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:20.453121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e9f3e-1b60-48d9-87fe-837b0abb2924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:20.453717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e9f3e-1b60-48d9-87fe-837b0abb2924 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:20.457330 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e9f3e-1b60-48d9-87fe-837b0abb2924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:20.459705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e9f3e-1b60-48d9-87fe-837b0abb2924 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:20.475628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:20.483351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1268/2488] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:49:20.487857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e9f3e-1b60-48d9-87fe-837b0abb2924 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:20.503879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e9f3e-1b60-48d9-87fe-837b0abb2924 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:20.511216 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd36a57-471f-40f6-8ebe-70360b0ee21f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:20.514068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c5b3a-e23f-4941-a9d9-e53bff7d8cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:20.515491 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c5b3a-e23f-4941-a9d9-e53bff7d8cf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:20.515647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c5b3a-e23f-4941-a9d9-e53bff7d8cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:20.516079 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c5b3a-e23f-4941-a9d9-e53bff7d8cf7 admin admin] PATH_INFO: `/v3/projects/9efa77222b784537858b15903d9d3b18/users/f822616b61634cec89bd87a9b06165cf/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:20.563724 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c5b3a-e23f-4941-a9d9-e53bff7d8cf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9efa77222b784537858b15903d9d3b18, user_id=f822616b61634cec89bd87a9b06165cf, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:20.568756 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c5b3a-e23f-4941-a9d9-e53bff7d8cf7 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:20.584191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:20.595490 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1269/2489] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:20 2022] PUT /identity/v3/projects/9efa77222b784537858b15903d9d3b18/users/f822616b61634cec89bd87a9b06165cf/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:20.612134 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd445ba-0141-4902-b903-5b193a66f40f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:20.612504 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd445ba-0141-4902-b903-5b193a66f40f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:20.612791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd445ba-0141-4902-b903-5b193a66f40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:20.679714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-6bd445ba-0141-4902-b903-5b193a66f40f None None] MFA Rules not processed for user `f822616b61634cec89bd87a9b06165cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:20.680996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03df48-2125-4476-b142-42c0e8e3aabc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:20.681446 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6bd445ba-0141-4902-b903-5b193a66f40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:20.683481 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03df48-2125-4476-b142-42c0e8e3aabc service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:20.685635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03df48-2125-4476-b142-42c0e8e3aabc service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:20.686257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03df48-2125-4476-b142-42c0e8e3aabc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:20.801839 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03df48-2125-4476-b142-42c0e8e3aabc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:20.803821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03df48-2125-4476-b142-42c0e8e3aabc service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:20.825899 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1270/2490] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:20.828020 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1221/2491] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:20 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:49:20.841601 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd445ba-0141-4902-b903-5b193a66f40f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:20.843196 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6bd445ba-0141-4902-b903-5b193a66f40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:21.016183 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd445ba-0141-4902-b903-5b193a66f40f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:21.027357 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c5fb2-d14f-4ae7-82e2-1a318785ea3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:21.028159 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c5fb2-d14f-4ae7-82e2-1a318785ea3c service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:21.028514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c5fb2-d14f-4ae7-82e2-1a318785ea3c service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:21.028709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c5fb2-d14f-4ae7-82e2-1a318785ea3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:21.041992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c5fb2-d14f-4ae7-82e2-1a318785ea3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:21.044768 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c5fb2-d14f-4ae7-82e2-1a318785ea3c service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:21.081395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1271/2492] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:21.527415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03df48-2125-4476-b142-42c0e8e3aabc service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:21.533397 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c5fb2-d14f-4ae7-82e2-1a318785ea3c service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:21.547004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03df48-2125-4476-b142-42c0e8e3aabc service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:21.598977 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c5fb2-d14f-4ae7-82e2-1a318785ea3c service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:21.604187 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070860b-1008-446e-90e8-ad2684fb1099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:21.604591 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37509ead-8ec0-4408-919a-d77824e63b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:21.605214 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070860b-1008-446e-90e8-ad2684fb1099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:21.605457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37509ead-8ec0-4408-919a-d77824e63b52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:21.605608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070860b-1008-446e-90e8-ad2684fb1099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:21.605764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37509ead-8ec0-4408-919a-d77824e63b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:21.605886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070860b-1008-446e-90e8-ad2684fb1099 admin admin] PATH_INFO: `/v3/users/c3af833b4cbb407e8530a1eee20237eb` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:21.606124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37509ead-8ec0-4408-919a-d77824e63b52 admin admin] PATH_INFO: `/v3/users/f822616b61634cec89bd87a9b06165cf` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:21.607090 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070860b-1008-446e-90e8-ad2684fb1099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3af833b4cbb407e8530a1eee20237eb)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:21.607255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37509ead-8ec0-4408-919a-d77824e63b52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f822616b61634cec89bd87a9b06165cf)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:21.609048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070860b-1008-446e-90e8-ad2684fb1099 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:21.609335 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37509ead-8ec0-4408-919a-d77824e63b52 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:21.625555 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:21.660016 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:21.662354 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4070860b-1008-446e-90e8-ad2684fb1099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3af833b4cbb407e8530a1eee20237eb'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:21.662735 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-37509ead-8ec0-4408-919a-d77824e63b52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f822616b61634cec89bd87a9b06165cf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:21.663314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4070860b-1008-446e-90e8-ad2684fb1099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3af833b4cbb407e8530a1eee20237eb'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:21.664080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4070860b-1008-446e-90e8-ad2684fb1099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3af833b4cbb407e8530a1eee20237eb'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:21.676810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-37509ead-8ec0-4408-919a-d77824e63b52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f822616b61634cec89bd87a9b06165cf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:21.677882 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4070860b-1008-446e-90e8-ad2684fb1099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3af833b4cbb407e8530a1eee20237eb'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:21.684206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-37509ead-8ec0-4408-919a-d77824e63b52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f822616b61634cec89bd87a9b06165cf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:21.684888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-37509ead-8ec0-4408-919a-d77824e63b52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f822616b61634cec89bd87a9b06165cf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:21.685517 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4070860b-1008-446e-90e8-ad2684fb1099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3af833b4cbb407e8530a1eee20237eb'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:21.692648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-37509ead-8ec0-4408-919a-d77824e63b52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f822616b61634cec89bd87a9b06165cf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:21.701702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1222/2493] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:21 2022] DELETE /identity/v3/users/f822616b61634cec89bd87a9b06165cf => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:21.704852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1272/2494] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:21 2022] DELETE /identity/v3/users/c3af833b4cbb407e8530a1eee20237eb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:21.708956 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5a356-5121-467d-93b5-416c9bb3e394 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:21.709302 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5a356-5121-467d-93b5-416c9bb3e394 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:21.709638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5a356-5121-467d-93b5-416c9bb3e394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:21.710065 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cf6d5-96c8-4f83-9fe2-354681b053c1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:21.710366 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cf6d5-96c8-4f83-9fe2-354681b053c1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:21.710565 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cf6d5-96c8-4f83-9fe2-354681b053c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:21.728956 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1c1cf6d5-96c8-4f83-9fe2-354681b053c1 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:21.730196 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1c1cf6d5-96c8-4f83-9fe2-354681b053c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:21.732462 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-28a5a356-5121-467d-93b5-416c9bb3e394 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:21.733628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-28a5a356-5121-467d-93b5-416c9bb3e394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:21.786902 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1273/2495] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:21.796047 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1223/2496] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:21.798935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cf6d5-96c8-4f83-9fe2-354681b053c1 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:21.800298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1c1cf6d5-96c8-4f83-9fe2-354681b053c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:21.804803 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5a356-5121-467d-93b5-416c9bb3e394 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:21.806922 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-28a5a356-5121-467d-93b5-416c9bb3e394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:21.857935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cf6d5-96c8-4f83-9fe2-354681b053c1 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:21.861536 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce493b-66c8-4541-8fbd-0ff169b9c359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:21.862218 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce493b-66c8-4541-8fbd-0ff169b9c359 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:21.862508 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce493b-66c8-4541-8fbd-0ff169b9c359 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:21.862787 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce493b-66c8-4541-8fbd-0ff169b9c359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:21.872698 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5a356-5121-467d-93b5-416c9bb3e394 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:21.872860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce493b-66c8-4541-8fbd-0ff169b9c359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:21.874284 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce493b-66c8-4541-8fbd-0ff169b9c359 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:21.876600 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8d023-bc09-424c-a989-1da870eb1a47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:21.877495 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8d023-bc09-424c-a989-1da870eb1a47 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:21.877774 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8d023-bc09-424c-a989-1da870eb1a47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:21.878068 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8d023-bc09-424c-a989-1da870eb1a47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:21.884619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1274/2497] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:49:21.887778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8d023-bc09-424c-a989-1da870eb1a47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:21.889198 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8d023-bc09-424c-a989-1da870eb1a47 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:21.898900 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1224/2498] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:49:22.359295 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce493b-66c8-4541-8fbd-0ff169b9c359 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:22.360734 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a3ce493b-66c8-4541-8fbd-0ff169b9c359 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:22.370483 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8d023-bc09-424c-a989-1da870eb1a47 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:22.371979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-90b8d023-bc09-424c-a989-1da870eb1a47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:22.381549 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce493b-66c8-4541-8fbd-0ff169b9c359 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:22.386744 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8d023-bc09-424c-a989-1da870eb1a47 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:22.387684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28acedd-47db-422e-9405-9ce3fe0be4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:22.388746 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28acedd-47db-422e-9405-9ce3fe0be4d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:22.389394 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28acedd-47db-422e-9405-9ce3fe0be4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:22.390073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28acedd-47db-422e-9405-9ce3fe0be4d5 admin admin] PATH_INFO: `/v3/projects/5f640a3e00c6443297ae50ea3dfe2240` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:22.391881 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3039e-afa1-40e1-8502-dc3bc70575bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:22.392082 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28acedd-47db-422e-9405-9ce3fe0be4d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f640a3e00c6443297ae50ea3dfe2240)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:22.392883 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3039e-afa1-40e1-8502-dc3bc70575bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:22.393262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3039e-afa1-40e1-8502-dc3bc70575bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:22.393630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3039e-afa1-40e1-8502-dc3bc70575bd admin admin] PATH_INFO: `/v3/projects/9efa77222b784537858b15903d9d3b18` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:22.394898 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28acedd-47db-422e-9405-9ce3fe0be4d5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:22.395038 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3039e-afa1-40e1-8502-dc3bc70575bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9efa77222b784537858b15903d9d3b18)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:22.397749 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3039e-afa1-40e1-8502-dc3bc70575bd admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:22.453905 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:22.471347 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b28acedd-47db-422e-9405-9ce3fe0be4d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f640a3e00c6443297ae50ea3dfe2240', 'request_id': 'req-b28acedd-47db-422e-9405-9ce3fe0be4d5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:22.476405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b0a3039e-afa1-40e1-8502-dc3bc70575bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9efa77222b784537858b15903d9d3b18', 'request_id': 'req-b0a3039e-afa1-40e1-8502-dc3bc70575bd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:22.480708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b28acedd-47db-422e-9405-9ce3fe0be4d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f640a3e00c6443297ae50ea3dfe2240', 'request_id': 'req-b28acedd-47db-422e-9405-9ce3fe0be4d5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:22.484493 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b0a3039e-afa1-40e1-8502-dc3bc70575bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9efa77222b784537858b15903d9d3b18', 'request_id': 'req-b0a3039e-afa1-40e1-8502-dc3bc70575bd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:22.487513 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b28acedd-47db-422e-9405-9ce3fe0be4d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f640a3e00c6443297ae50ea3dfe2240', 'request_id': 'req-b28acedd-47db-422e-9405-9ce3fe0be4d5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:22.487780 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:22.500546 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b28acedd-47db-422e-9405-9ce3fe0be4d5 admin admin] The token cache is being invalidate because project 5f640a3e00c6443297ae50ea3dfe2240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:22.500837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b28acedd-47db-422e-9405-9ce3fe0be4d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:22.501508 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b28acedd-47db-422e-9405-9ce3fe0be4d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:22.502624 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b0a3039e-afa1-40e1-8502-dc3bc70575bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9efa77222b784537858b15903d9d3b18', 'request_id': 'req-b0a3039e-afa1-40e1-8502-dc3bc70575bd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:22.503557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1275/2499] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:22 2022] DELETE /identity/v3/projects/5f640a3e00c6443297ae50ea3dfe2240 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:22.507150 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28acedd-47db-422e-9405-9ce3fe0be4d5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:22.510243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b28acedd-47db-422e-9405-9ce3fe0be4d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:22.516953 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b0a3039e-afa1-40e1-8502-dc3bc70575bd admin admin] The token cache is being invalidate because project 9efa77222b784537858b15903d9d3b18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:22.517246 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b0a3039e-afa1-40e1-8502-dc3bc70575bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:22.517916 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b0a3039e-afa1-40e1-8502-dc3bc70575bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:22.519700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1225/2500] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:22 2022] DELETE /identity/v3/projects/9efa77222b784537858b15903d9d3b18 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:22.524606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee06db-2bcb-4941-b51b-cffc031fd276 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:22.524787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee06db-2bcb-4941-b51b-cffc031fd276 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:22.525183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee06db-2bcb-4941-b51b-cffc031fd276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:22.544293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-22ee06db-2bcb-4941-b51b-cffc031fd276 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:22.545520 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-22ee06db-2bcb-4941-b51b-cffc031fd276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:22.588368 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28acedd-47db-422e-9405-9ce3fe0be4d5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:22.609984 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1226/2501] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:22.618403 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774d876-78fa-45b4-8b1a-cee8034ed02c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:22.618755 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774d876-78fa-45b4-8b1a-cee8034ed02c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:22.619095 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774d876-78fa-45b4-8b1a-cee8034ed02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:22.643242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-5774d876-78fa-45b4-8b1a-cee8034ed02c None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:22.644463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5774d876-78fa-45b4-8b1a-cee8034ed02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:22.650109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573635ec-2d98-4244-8680-0afef1e33192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:22.651036 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573635ec-2d98-4244-8680-0afef1e33192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:22.651329 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573635ec-2d98-4244-8680-0afef1e33192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:22.651620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573635ec-2d98-4244-8680-0afef1e33192 admin admin] PATH_INFO: `/v3/users/5c7a6d39ae054f52b40350018ada7bd8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:22.653024 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573635ec-2d98-4244-8680-0afef1e33192 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c7a6d39ae054f52b40350018ada7bd8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:22.653313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1227/2502] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:49:22.657244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573635ec-2d98-4244-8680-0afef1e33192 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:22.659311 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774d876-78fa-45b4-8b1a-cee8034ed02c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:22.679482 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774d876-78fa-45b4-8b1a-cee8034ed02c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:22.684453 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86273a4a-a989-40d7-8e1b-fd1315a813ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:22.685121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86273a4a-a989-40d7-8e1b-fd1315a813ae admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:22.685403 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86273a4a-a989-40d7-8e1b-fd1315a813ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:22.685690 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86273a4a-a989-40d7-8e1b-fd1315a813ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:22.686616 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86273a4a-a989-40d7-8e1b-fd1315a813ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:49:22.687084 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86273a4a-a989-40d7-8e1b-fd1315a813ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:22.689316 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86273a4a-a989-40d7-8e1b-fd1315a813ae admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:22.703783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1228/2503] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:49:22.709497 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86273a4a-a989-40d7-8e1b-fd1315a813ae admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:22.717421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-573635ec-2d98-4244-8680-0afef1e33192 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c7a6d39ae054f52b40350018ada7bd8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:22.718136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-573635ec-2d98-4244-8680-0afef1e33192 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c7a6d39ae054f52b40350018ada7bd8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:22.718792 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-573635ec-2d98-4244-8680-0afef1e33192 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c7a6d39ae054f52b40350018ada7bd8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:22.722253 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86273a4a-a989-40d7-8e1b-fd1315a813ae admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:22.727976 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f8f54-ea5a-4241-8376-57a27b02d084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:22.728785 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f8f54-ea5a-4241-8376-57a27b02d084 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:22.729237 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f8f54-ea5a-4241-8376-57a27b02d084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:22.729650 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f8f54-ea5a-4241-8376-57a27b02d084 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:22.730755 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f8f54-ea5a-4241-8376-57a27b02d084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:22.731648 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-573635ec-2d98-4244-8680-0afef1e33192 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c7a6d39ae054f52b40350018ada7bd8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:22.733784 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f8f54-ea5a-4241-8376-57a27b02d084 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:22.736143 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:22.739737 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-573635ec-2d98-4244-8680-0afef1e33192 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c7a6d39ae054f52b40350018ada7bd8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:22.755261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1276/2504] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:22 2022] DELETE /identity/v3/users/5c7a6d39ae054f52b40350018ada7bd8 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:22.769693 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1229/2505] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:22 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:22.771162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573635ec-2d98-4244-8680-0afef1e33192 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:22.772687 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-573635ec-2d98-4244-8680-0afef1e33192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:22.840278 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573635ec-2d98-4244-8680-0afef1e33192 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:22.911765 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa664c-b134-4b4b-b4ea-4957083dbf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:22.912497 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa664c-b134-4b4b-b4ea-4957083dbf2b admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:22.912860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa664c-b134-4b4b-b4ea-4957083dbf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:22.913212 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa664c-b134-4b4b-b4ea-4957083dbf2b admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:22.914018 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa664c-b134-4b4b-b4ea-4957083dbf2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:22.916142 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa664c-b134-4b4b-b4ea-4957083dbf2b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:22.942111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1277/2506] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:22 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:22.943948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f8f54-ea5a-4241-8376-57a27b02d084 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:22.954793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f8f54-ea5a-4241-8376-57a27b02d084 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:22.959403 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc84b96-a475-41a1-bbb2-1b5e6f24c455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:22.960053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc84b96-a475-41a1-bbb2-1b5e6f24c455 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:22.960331 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc84b96-a475-41a1-bbb2-1b5e6f24c455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:22.960603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc84b96-a475-41a1-bbb2-1b5e6f24c455 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:22.961322 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc84b96-a475-41a1-bbb2-1b5e6f24c455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:22.963498 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc84b96-a475-41a1-bbb2-1b5e6f24c455 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:22.973633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1230/2507] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:22.977534 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa664c-b134-4b4b-b4ea-4957083dbf2b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:22.987542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa664c-b134-4b4b-b4ea-4957083dbf2b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:22.992213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf4faa-093c-474b-902a-47752530d8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:22.992893 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf4faa-093c-474b-902a-47752530d8ad admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:22.993263 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf4faa-093c-474b-902a-47752530d8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:22.993536 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf4faa-093c-474b-902a-47752530d8ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:22.994285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf4faa-093c-474b-902a-47752530d8ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:22.995880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf4faa-093c-474b-902a-47752530d8ad admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:23.006280 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1278/2508] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:23.010950 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc84b96-a475-41a1-bbb2-1b5e6f24c455 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:23.020819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc84b96-a475-41a1-bbb2-1b5e6f24c455 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:23.026638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634cd721-fdb1-478f-90c9-39ee6142a900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:23.027996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cd721-fdb1-478f-90c9-39ee6142a900 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.028155 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cd721-fdb1-478f-90c9-39ee6142a900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.028450 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cd721-fdb1-478f-90c9-39ee6142a900 admin admin] PATH_INFO: `/v3/projects/7f3523b1f598414687844fc588cc4c6c/users/0a44f07b6f384903bd2ad268a472f268/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.047700 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634cd721-fdb1-478f-90c9-39ee6142a900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f3523b1f598414687844fc588cc4c6c, user_id=0a44f07b6f384903bd2ad268a472f268, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:23.053622 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634cd721-fdb1-478f-90c9-39ee6142a900 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:23.054488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf4faa-093c-474b-902a-47752530d8ad admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:23.056566 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8eaf4faa-093c-474b-902a-47752530d8ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:23.072381 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1231/2509] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:23 2022] PUT /identity/v3/projects/7f3523b1f598414687844fc588cc4c6c/users/0a44f07b6f384903bd2ad268a472f268/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:23.074402 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf4faa-093c-474b-902a-47752530d8ad admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:23.093135 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae8c95-7278-41b9-8d39-4e3bc0954051 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.093389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae8c95-7278-41b9-8d39-4e3bc0954051 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.093751 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae8c95-7278-41b9-8d39-4e3bc0954051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.121951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-c9ae8c95-7278-41b9-8d39-4e3bc0954051 None None] MFA Rules not processed for user `0a44f07b6f384903bd2ad268a472f268`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:23.123280 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c9ae8c95-7278-41b9-8d39-4e3bc0954051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:23.148649 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3ec566-aecc-4fd1-85c8-c605a8bce15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:23.150103 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ec566-aecc-4fd1-85c8-c605a8bce15f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.150382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ec566-aecc-4fd1-85c8-c605a8bce15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.150655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ec566-aecc-4fd1-85c8-c605a8bce15f admin admin] PATH_INFO: `/v3/projects/13f533a2230b48ef9aabf597e04bcc31` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.151842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3ec566-aecc-4fd1-85c8-c605a8bce15f admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f533a2230b48ef9aabf597e04bcc31)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:23.154405 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3ec566-aecc-4fd1-85c8-c605a8bce15f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:23.170654 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:23.189799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1232/2510] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:23.194195 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae8c95-7278-41b9-8d39-4e3bc0954051 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:23.199873 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:23.203911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2a3ec566-aecc-4fd1-85c8-c605a8bce15f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f533a2230b48ef9aabf597e04bcc31', 'request_id': 'req-2a3ec566-aecc-4fd1-85c8-c605a8bce15f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:23.205685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae8c95-7278-41b9-8d39-4e3bc0954051 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:23.212848 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2a3ec566-aecc-4fd1-85c8-c605a8bce15f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f533a2230b48ef9aabf597e04bcc31', 'request_id': 'req-2a3ec566-aecc-4fd1-85c8-c605a8bce15f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:23.219365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2a3ec566-aecc-4fd1-85c8-c605a8bce15f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f533a2230b48ef9aabf597e04bcc31', 'request_id': 'req-2a3ec566-aecc-4fd1-85c8-c605a8bce15f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:23.229589 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2a3ec566-aecc-4fd1-85c8-c605a8bce15f admin admin] The token cache is being invalidate because project 13f533a2230b48ef9aabf597e04bcc31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:23.229879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2a3ec566-aecc-4fd1-85c8-c605a8bce15f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:23.230563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2a3ec566-aecc-4fd1-85c8-c605a8bce15f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:23.232125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1279/2511] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:23 2022] DELETE /identity/v3/projects/13f533a2230b48ef9aabf597e04bcc31 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:23.236938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddf254-cdb5-42c4-a49a-da1783864d07 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.237225 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddf254-cdb5-42c4-a49a-da1783864d07 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.237496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddf254-cdb5-42c4-a49a-da1783864d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.256176 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d5ddf254-cdb5-42c4-a49a-da1783864d07 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:23.257371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d5ddf254-cdb5-42c4-a49a-da1783864d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:23.267930 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e91b7-e8dd-418c-af10-b48381c00475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:23.268644 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e91b7-e8dd-418c-af10-b48381c00475 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.269618 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e91b7-e8dd-418c-af10-b48381c00475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.269905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e91b7-e8dd-418c-af10-b48381c00475 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.270732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e91b7-e8dd-418c-af10-b48381c00475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:23.272614 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e91b7-e8dd-418c-af10-b48381c00475 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:23.300368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1233/2512] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:23 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:49:23.304130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e91b7-e8dd-418c-af10-b48381c00475 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:23.306413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-597e91b7-e8dd-418c-af10-b48381c00475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:23.318047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1280/2513] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:49:23.327729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7a7ee-92f9-4ae8-9026-c270439c496f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.328079 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7a7ee-92f9-4ae8-9026-c270439c496f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.328301 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7a7ee-92f9-4ae8-9026-c270439c496f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.351593 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-0ab7a7ee-92f9-4ae8-9026-c270439c496f None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:23.353604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0ab7a7ee-92f9-4ae8-9026-c270439c496f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:23.374613 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e91b7-e8dd-418c-af10-b48381c00475 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:23.380082 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f385b59-c6ee-45ae-b3f1-5774992d912f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:23.380948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f385b59-c6ee-45ae-b3f1-5774992d912f admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.381326 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f385b59-c6ee-45ae-b3f1-5774992d912f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.381696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f385b59-c6ee-45ae-b3f1-5774992d912f admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.382543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f385b59-c6ee-45ae-b3f1-5774992d912f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:23.384675 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f385b59-c6ee-45ae-b3f1-5774992d912f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:23.411555 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1234/2514] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:23 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:49:23.412764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f385b59-c6ee-45ae-b3f1-5774992d912f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:23.427520 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f385b59-c6ee-45ae-b3f1-5774992d912f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:23.432189 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc249a9-a5d8-45f6-9dbf-a5a61e0c249a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:23.432866 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc249a9-a5d8-45f6-9dbf-a5a61e0c249a admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.433128 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc249a9-a5d8-45f6-9dbf-a5a61e0c249a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.433396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc249a9-a5d8-45f6-9dbf-a5a61e0c249a admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.434148 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc249a9-a5d8-45f6-9dbf-a5a61e0c249a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:23.435140 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1281/2515] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:23.438808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc249a9-a5d8-45f6-9dbf-a5a61e0c249a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:23.439281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7a7ee-92f9-4ae8-9026-c270439c496f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:23.448288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7a7ee-92f9-4ae8-9026-c270439c496f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:23.452023 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1235/2516] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:23.452977 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc249a9-a5d8-45f6-9dbf-a5a61e0c249a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:23.453384 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530875d5-7f6d-4cc2-bad9-b5e521dfd7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:23.454090 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530875d5-7f6d-4cc2-bad9-b5e521dfd7bf admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.454381 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530875d5-7f6d-4cc2-bad9-b5e521dfd7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.454660 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530875d5-7f6d-4cc2-bad9-b5e521dfd7bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.455392 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530875d5-7f6d-4cc2-bad9-b5e521dfd7bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:49:23.455706 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530875d5-7f6d-4cc2-bad9-b5e521dfd7bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:23.457826 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530875d5-7f6d-4cc2-bad9-b5e521dfd7bf admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:23.462922 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc249a9-a5d8-45f6-9dbf-a5a61e0c249a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:23.467730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48044f5d-6c14-4c4c-a8bd-d7166b783cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:23.468626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48044f5d-6c14-4c4c-a8bd-d7166b783cac admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.468910 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48044f5d-6c14-4c4c-a8bd-d7166b783cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.469184 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48044f5d-6c14-4c4c-a8bd-d7166b783cac admin admin] PATH_INFO: `/v3/projects/7db3fed1b7fb46f19dac32dea83028e1/users/ae13c980207b4ba2a6bf5c8d666c84a0/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.474152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1282/2517] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:49:23.475054 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530875d5-7f6d-4cc2-bad9-b5e521dfd7bf admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:23.485675 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530875d5-7f6d-4cc2-bad9-b5e521dfd7bf admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:23.488620 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48044f5d-6c14-4c4c-a8bd-d7166b783cac admin admin] RBAC: Authorizing `identity:create_grant(project_id=7db3fed1b7fb46f19dac32dea83028e1, user_id=ae13c980207b4ba2a6bf5c8d666c84a0, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:23.490430 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ba997-76c7-4e18-8f9e-f8fd036e475f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:23.491073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ba997-76c7-4e18-8f9e-f8fd036e475f admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.491359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ba997-76c7-4e18-8f9e-f8fd036e475f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.491458 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48044f5d-6c14-4c4c-a8bd-d7166b783cac admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:23.491668 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ba997-76c7-4e18-8f9e-f8fd036e475f admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.492351 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ba997-76c7-4e18-8f9e-f8fd036e475f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:23.494352 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ba997-76c7-4e18-8f9e-f8fd036e475f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:23.505541 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1236/2518] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:23 2022] PUT /identity/v3/projects/7db3fed1b7fb46f19dac32dea83028e1/users/ae13c980207b4ba2a6bf5c8d666c84a0/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:23.509413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48044f5d-6c14-4c4c-a8bd-d7166b783cac admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:23.524720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1283/2519] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:23 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:49:23.525650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ba997-76c7-4e18-8f9e-f8fd036e475f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:23.534978 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:23.559848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:23.584348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48044f5d-6c14-4c4c-a8bd-d7166b783cac admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:23.599168 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ba997-76c7-4e18-8f9e-f8fd036e475f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:23.604073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9302ff6-ecf8-4d8d-807b-8cc4a11e76df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:23.604809 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9302ff6-ecf8-4d8d-807b-8cc4a11e76df admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.605165 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9302ff6-ecf8-4d8d-807b-8cc4a11e76df admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.605507 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9302ff6-ecf8-4d8d-807b-8cc4a11e76df admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.606298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9302ff6-ecf8-4d8d-807b-8cc4a11e76df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:23.608305 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9302ff6-ecf8-4d8d-807b-8cc4a11e76df admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:23.628548 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1284/2520] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:23 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:49:23.632324 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9302ff6-ecf8-4d8d-807b-8cc4a11e76df admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:23.643896 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9302ff6-ecf8-4d8d-807b-8cc4a11e76df admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:23.647930 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b383fa7-59f2-41d0-b67c-25fcd211ade6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:23.648628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b383fa7-59f2-41d0-b67c-25fcd211ade6 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.648906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b383fa7-59f2-41d0-b67c-25fcd211ade6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.649864 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b383fa7-59f2-41d0-b67c-25fcd211ade6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.650075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175382f-c905-499e-865a-8fa682d2d419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:23.650692 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b383fa7-59f2-41d0-b67c-25fcd211ade6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:23.650891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175382f-c905-499e-865a-8fa682d2d419 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.651141 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175382f-c905-499e-865a-8fa682d2d419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.651428 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175382f-c905-499e-865a-8fa682d2d419 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.652156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175382f-c905-499e-865a-8fa682d2d419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:23.652264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b383fa7-59f2-41d0-b67c-25fcd211ade6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:23.654154 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175382f-c905-499e-865a-8fa682d2d419 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:23.663765 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1237/2521] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:23.667960 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b383fa7-59f2-41d0-b67c-25fcd211ade6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:23.671873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1285/2522] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:49:23.672893 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175382f-c905-499e-865a-8fa682d2d419 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:23.679946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b383fa7-59f2-41d0-b67c-25fcd211ade6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:23.683590 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175382f-c905-499e-865a-8fa682d2d419 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:23.685474 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5bb80-e84e-474b-b1b3-7217ba148c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:23.686566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5bb80-e84e-474b-b1b3-7217ba148c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.686945 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5bb80-e84e-474b-b1b3-7217ba148c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.687320 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5bb80-e84e-474b-b1b3-7217ba148c12 admin admin] PATH_INFO: `/v3/projects/7db3fed1b7fb46f19dac32dea83028e1/users/ae13c980207b4ba2a6bf5c8d666c84a0/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.688452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b24f70-ac2e-4595-9417-d57c990d9372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:23.689126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b24f70-ac2e-4595-9417-d57c990d9372 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.689588 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b24f70-ac2e-4595-9417-d57c990d9372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.689721 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5bb80-e84e-474b-b1b3-7217ba148c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7db3fed1b7fb46f19dac32dea83028e1, user_id=ae13c980207b4ba2a6bf5c8d666c84a0, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:23.689871 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b24f70-ac2e-4595-9417-d57c990d9372 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.690639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b24f70-ac2e-4595-9417-d57c990d9372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:23.692259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b24f70-ac2e-4595-9417-d57c990d9372 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:23.692733 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5bb80-e84e-474b-b1b3-7217ba148c12 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:23.704798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1238/2523] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:23 2022] PUT /identity/v3/projects/7db3fed1b7fb46f19dac32dea83028e1/users/ae13c980207b4ba2a6bf5c8d666c84a0/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:23.707969 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1286/2524] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:23.716212 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5bb80-e84e-474b-b1b3-7217ba148c12 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:23.716756 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c157c9f-d9bb-494a-8368-26255b3e9825 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.717075 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c157c9f-d9bb-494a-8368-26255b3e9825 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.717359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c157c9f-d9bb-494a-8368-26255b3e9825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.748244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1c157c9f-d9bb-494a-8368-26255b3e9825 None None] MFA Rules not processed for user `ae13c980207b4ba2a6bf5c8d666c84a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:23.749588 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1c157c9f-d9bb-494a-8368-26255b3e9825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:23.790466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5bb80-e84e-474b-b1b3-7217ba148c12 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:23.796626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efb828-902a-44ee-b0c7-33d457b7d099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:23.797635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efb828-902a-44ee-b0c7-33d457b7d099 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.798021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efb828-902a-44ee-b0c7-33d457b7d099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.798390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efb828-902a-44ee-b0c7-33d457b7d099 admin admin] PATH_INFO: `/v3/projects/0b81af8c96e64846a84740d38513965f/users/f516de59ae5a4adca5826185e5156d76/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.817410 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efb828-902a-44ee-b0c7-33d457b7d099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b81af8c96e64846a84740d38513965f, user_id=f516de59ae5a4adca5826185e5156d76, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:23.820209 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efb828-902a-44ee-b0c7-33d457b7d099 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:23.839251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1287/2525] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:49:23.845868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1239/2526] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:23 2022] PUT /identity/v3/projects/0b81af8c96e64846a84740d38513965f/users/f516de59ae5a4adca5826185e5156d76/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:23.850461 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c157c9f-d9bb-494a-8368-26255b3e9825 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:23.852237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1c157c9f-d9bb-494a-8368-26255b3e9825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:23.852878 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2de6c-dab4-4bd7-8ebd-31a7b9e2c00f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.853173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2de6c-dab4-4bd7-8ebd-31a7b9e2c00f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.853456 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2de6c-dab4-4bd7-8ebd-31a7b9e2c00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:23.879527 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9ad2de6c-dab4-4bd7-8ebd-31a7b9e2c00f None None] MFA Rules not processed for user `f516de59ae5a4adca5826185e5156d76`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:23.880769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9ad2de6c-dab4-4bd7-8ebd-31a7b9e2c00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:23.908919 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c157c9f-d9bb-494a-8368-26255b3e9825 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:23.920761 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:23.927944 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:23.950138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1240/2527] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:49:23.954374 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2de6c-dab4-4bd7-8ebd-31a7b9e2c00f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:23.961456 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b01c04-0569-44d8-99ae-ae85dd3eea2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:23.963018 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b01c04-0569-44d8-99ae-ae85dd3eea2b service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:23.963380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b01c04-0569-44d8-99ae-ae85dd3eea2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:23.963662 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b01c04-0569-44d8-99ae-ae85dd3eea2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:24.016859 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2de6c-dab4-4bd7-8ebd-31a7b9e2c00f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:24.021539 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987ef11-d3a0-4981-ac8a-5c4622e865f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:24.022236 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987ef11-d3a0-4981-ac8a-5c4622e865f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:24.022518 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987ef11-d3a0-4981-ac8a-5c4622e865f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:24.022796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987ef11-d3a0-4981-ac8a-5c4622e865f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:24.023484 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987ef11-d3a0-4981-ac8a-5c4622e865f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:24.025467 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987ef11-d3a0-4981-ac8a-5c4622e865f4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:24.030961 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b01c04-0569-44d8-99ae-ae85dd3eea2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:24.032696 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b01c04-0569-44d8-99ae-ae85dd3eea2b service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:24.045039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1288/2528] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:24.060907 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1241/2529] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:23 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:49:24.065140 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b01c04-0569-44d8-99ae-ae85dd3eea2b service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:24.077822 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b01c04-0569-44d8-99ae-ae85dd3eea2b service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:24.136962 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae519d-e022-4334-94bd-1ff6eaeb8ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:24.137671 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae519d-e022-4334-94bd-1ff6eaeb8ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:24.137975 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae519d-e022-4334-94bd-1ff6eaeb8ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:24.138271 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae519d-e022-4334-94bd-1ff6eaeb8ca5 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:24.138964 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae519d-e022-4334-94bd-1ff6eaeb8ca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:24.141012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae519d-e022-4334-94bd-1ff6eaeb8ca5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:24.164579 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1289/2530] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:24 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:24.165670 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987ef11-d3a0-4981-ac8a-5c4622e865f4 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:24.176449 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987ef11-d3a0-4981-ac8a-5c4622e865f4 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:24.184431 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c2569-a4f4-48b8-8ec5-5912fbb9caf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:24.185099 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c2569-a4f4-48b8-8ec5-5912fbb9caf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:24.185386 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c2569-a4f4-48b8-8ec5-5912fbb9caf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:24.185667 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c2569-a4f4-48b8-8ec5-5912fbb9caf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:24.186435 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c2569-a4f4-48b8-8ec5-5912fbb9caf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:24.188032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c2569-a4f4-48b8-8ec5-5912fbb9caf0 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:24.199998 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1242/2531] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:24.204127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae519d-e022-4334-94bd-1ff6eaeb8ca5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:24.214819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae519d-e022-4334-94bd-1ff6eaeb8ca5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:24.221180 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57027da-7ad2-4b73-a0f6-d2a4bf56ee91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:24.222435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57027da-7ad2-4b73-a0f6-d2a4bf56ee91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:24.222722 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57027da-7ad2-4b73-a0f6-d2a4bf56ee91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:24.223004 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57027da-7ad2-4b73-a0f6-d2a4bf56ee91 admin admin] PATH_INFO: `/v3/projects/16d10495cd9d4736a7410e4e69a9fb86/users/a7da6dbe8d06480281cdc07d5d987237/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:24.240023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57027da-7ad2-4b73-a0f6-d2a4bf56ee91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16d10495cd9d4736a7410e4e69a9fb86, user_id=a7da6dbe8d06480281cdc07d5d987237, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:24.243293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57027da-7ad2-4b73-a0f6-d2a4bf56ee91 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:24.267840 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1290/2532] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:24 2022] PUT /identity/v3/projects/16d10495cd9d4736a7410e4e69a9fb86/users/a7da6dbe8d06480281cdc07d5d987237/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:24.274804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c2569-a4f4-48b8-8ec5-5912fbb9caf0 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:24.289311 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c2569-a4f4-48b8-8ec5-5912fbb9caf0 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:24.353685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb2c75-f183-450e-9315-b05728997241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:24.354404 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb2c75-f183-450e-9315-b05728997241 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:24.354691 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb2c75-f183-450e-9315-b05728997241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:24.354968 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb2c75-f183-450e-9315-b05728997241 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:24.355807 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb2c75-f183-450e-9315-b05728997241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:24.357535 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb2c75-f183-450e-9315-b05728997241 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:24.373310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1243/2533] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:49:24.376459 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57027da-7ad2-4b73-a0f6-d2a4bf56ee91 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:24.387941 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57027da-7ad2-4b73-a0f6-d2a4bf56ee91 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:24.393346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afaf98-bf53-430f-ace3-110631838309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:24.394293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afaf98-bf53-430f-ace3-110631838309 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:24.394581 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afaf98-bf53-430f-ace3-110631838309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:24.394861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afaf98-bf53-430f-ace3-110631838309 admin admin] PATH_INFO: `/v3/projects/16d10495cd9d4736a7410e4e69a9fb86/users/a7da6dbe8d06480281cdc07d5d987237/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:24.396945 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afaf98-bf53-430f-ace3-110631838309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16d10495cd9d4736a7410e4e69a9fb86, user_id=a7da6dbe8d06480281cdc07d5d987237, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:24.400359 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afaf98-bf53-430f-ace3-110631838309 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:24.414832 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1291/2534] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:24 2022] PUT /identity/v3/projects/16d10495cd9d4736a7410e4e69a9fb86/users/a7da6dbe8d06480281cdc07d5d987237/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:24.425973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591abb13-b5f5-49e9-9813-4cf76f341dd1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:24.426286 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591abb13-b5f5-49e9-9813-4cf76f341dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:24.426570 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591abb13-b5f5-49e9-9813-4cf76f341dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:24.461589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-591abb13-b5f5-49e9-9813-4cf76f341dd1 None None] MFA Rules not processed for user `a7da6dbe8d06480281cdc07d5d987237`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:24.462597 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-591abb13-b5f5-49e9-9813-4cf76f341dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:24.483371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afaf98-bf53-430f-ace3-110631838309 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:24.506936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:24.516066 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:24.543524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1244/2535] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:24 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:24.550684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afaf98-bf53-430f-ace3-110631838309 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:24.554100 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591abb13-b5f5-49e9-9813-4cf76f341dd1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:24.556199 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c867562-3344-41b5-b9a3-c2b92ff65b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:24.557080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c867562-3344-41b5-b9a3-c2b92ff65b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:24.557368 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c867562-3344-41b5-b9a3-c2b92ff65b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:24.557650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c867562-3344-41b5-b9a3-c2b92ff65b7d admin admin] PATH_INFO: `/v3/users/0a44f07b6f384903bd2ad268a472f268` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:24.558925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c867562-3344-41b5-b9a3-c2b92ff65b7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a44f07b6f384903bd2ad268a472f268)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:24.560869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c867562-3344-41b5-b9a3-c2b92ff65b7d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:24.610364 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1c867562-3344-41b5-b9a3-c2b92ff65b7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a44f07b6f384903bd2ad268a472f268'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:24.611052 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1c867562-3344-41b5-b9a3-c2b92ff65b7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a44f07b6f384903bd2ad268a472f268'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:24.611664 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1c867562-3344-41b5-b9a3-c2b92ff65b7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a44f07b6f384903bd2ad268a472f268'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:24.612898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591abb13-b5f5-49e9-9813-4cf76f341dd1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:24.617268 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9946cd-eb05-4efb-aa5e-3be3eebbefa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:24.617982 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9946cd-eb05-4efb-aa5e-3be3eebbefa6 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:24.618781 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9946cd-eb05-4efb-aa5e-3be3eebbefa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:24.619056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9946cd-eb05-4efb-aa5e-3be3eebbefa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:24.620949 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ae9946cd-eb05-4efb-aa5e-3be3eebbefa6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:24.625397 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1c867562-3344-41b5-b9a3-c2b92ff65b7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a44f07b6f384903bd2ad268a472f268'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:24.641319 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9946cd-eb05-4efb-aa5e-3be3eebbefa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:24.642189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1c867562-3344-41b5-b9a3-c2b92ff65b7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a44f07b6f384903bd2ad268a472f268'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:24.642815 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9946cd-eb05-4efb-aa5e-3be3eebbefa6 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:24.655200 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1245/2536] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:24 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:49:24.659728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1292/2537] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:24 2022] DELETE /identity/v3/users/0a44f07b6f384903bd2ad268a472f268 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:24.664518 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e845dda-fa54-4cc5-93a9-c46580cf9bdf None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:24.665417 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e845dda-fa54-4cc5-93a9-c46580cf9bdf None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:24.665896 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e845dda-fa54-4cc5-93a9-c46580cf9bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:24.686845 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7e845dda-fa54-4cc5-93a9-c46580cf9bdf None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:24.688333 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7e845dda-fa54-4cc5-93a9-c46580cf9bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:24.758248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1246/2538] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:49:24.769717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c867562-3344-41b5-b9a3-c2b92ff65b7d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:24.771193 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1c867562-3344-41b5-b9a3-c2b92ff65b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:24.839728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c867562-3344-41b5-b9a3-c2b92ff65b7d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:24.844068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba539d21-c814-4220-a3ed-81433766200c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:24.844727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba539d21-c814-4220-a3ed-81433766200c service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:24.845298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba539d21-c814-4220-a3ed-81433766200c service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:24.845298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba539d21-c814-4220-a3ed-81433766200c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:24.856133 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba539d21-c814-4220-a3ed-81433766200c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:24.857604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba539d21-c814-4220-a3ed-81433766200c service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:24.869448 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1293/2539] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:25.326809 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e845dda-fa54-4cc5-93a9-c46580cf9bdf None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:25.330786 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7e845dda-fa54-4cc5-93a9-c46580cf9bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:25.350252 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e845dda-fa54-4cc5-93a9-c46580cf9bdf None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:25.354849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:25.355674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:25.355977 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:25.356269 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5 admin admin] PATH_INFO: `/v3/projects/7f3523b1f598414687844fc588cc4c6c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:25.357397 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f3523b1f598414687844fc588cc4c6c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:25.359696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:25.409471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f3523b1f598414687844fc588cc4c6c', 'request_id': 'req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:25.418790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f3523b1f598414687844fc588cc4c6c', 'request_id': 'req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:25.427845 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f3523b1f598414687844fc588cc4c6c', 'request_id': 'req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:25.442203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5 admin admin] The token cache is being invalidate because project 7f3523b1f598414687844fc588cc4c6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:25.442513 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:25.443345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:25.444746 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1247/2540] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:25 2022] DELETE /identity/v3/projects/7f3523b1f598414687844fc588cc4c6c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:25.448987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba539d21-c814-4220-a3ed-81433766200c service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:25.459771 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba539d21-c814-4220-a3ed-81433766200c service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:25.524562 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:25.550271 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2034ea-4a57-486c-ba43-167a4ea0833b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:25.551277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2034ea-4a57-486c-ba43-167a4ea0833b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:25.551562 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2034ea-4a57-486c-ba43-167a4ea0833b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:25.551842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2034ea-4a57-486c-ba43-167a4ea0833b admin admin] PATH_INFO: `/v3/users/ae13c980207b4ba2a6bf5c8d666c84a0` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:25.553124 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2034ea-4a57-486c-ba43-167a4ea0833b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae13c980207b4ba2a6bf5c8d666c84a0)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:25.555877 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2034ea-4a57-486c-ba43-167a4ea0833b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:25.610065 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-cc2034ea-4a57-486c-ba43-167a4ea0833b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae13c980207b4ba2a6bf5c8d666c84a0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:25.610913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-cc2034ea-4a57-486c-ba43-167a4ea0833b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae13c980207b4ba2a6bf5c8d666c84a0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:25.611514 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-cc2034ea-4a57-486c-ba43-167a4ea0833b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae13c980207b4ba2a6bf5c8d666c84a0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:25.624540 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-cc2034ea-4a57-486c-ba43-167a4ea0833b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae13c980207b4ba2a6bf5c8d666c84a0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:25.632168 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-cc2034ea-4a57-486c-ba43-167a4ea0833b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae13c980207b4ba2a6bf5c8d666c84a0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:25.645259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1294/2541] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:25 2022] DELETE /identity/v3/users/ae13c980207b4ba2a6bf5c8d666c84a0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:25.955947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:25.957659 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:25.972607 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe598d-4eba-41a7-b794-e1dd54dd2fd5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:25.979800 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:26.037911 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad5826-3afc-4604-bb15-31b0be534d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:26.038813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad5826-3afc-4604-bb15-31b0be534d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:26.039101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad5826-3afc-4604-bb15-31b0be534d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:26.039498 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad5826-3afc-4604-bb15-31b0be534d08 admin admin] PATH_INFO: `/v3/projects/7db3fed1b7fb46f19dac32dea83028e1` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:26.040923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad5826-3afc-4604-bb15-31b0be534d08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7db3fed1b7fb46f19dac32dea83028e1)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:26.042906 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad5826-3afc-4604-bb15-31b0be534d08 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:26.095454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0fad5826-3afc-4604-bb15-31b0be534d08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7db3fed1b7fb46f19dac32dea83028e1', 'request_id': 'req-0fad5826-3afc-4604-bb15-31b0be534d08'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:26.106863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0fad5826-3afc-4604-bb15-31b0be534d08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7db3fed1b7fb46f19dac32dea83028e1', 'request_id': 'req-0fad5826-3afc-4604-bb15-31b0be534d08'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:26.115371 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0fad5826-3afc-4604-bb15-31b0be534d08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7db3fed1b7fb46f19dac32dea83028e1', 'request_id': 'req-0fad5826-3afc-4604-bb15-31b0be534d08'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:26.129601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0fad5826-3afc-4604-bb15-31b0be534d08 admin admin] The token cache is being invalidate because project 7db3fed1b7fb46f19dac32dea83028e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:26.129913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0fad5826-3afc-4604-bb15-31b0be534d08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:26.130605 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0fad5826-3afc-4604-bb15-31b0be534d08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:26.132584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1248/2542] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:25 2022] DELETE /identity/v3/projects/7db3fed1b7fb46f19dac32dea83028e1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:26.138970 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddc1b8-b725-47dd-aa4c-4ea59928d878 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:26.139377 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddc1b8-b725-47dd-aa4c-4ea59928d878 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:26.139766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddc1b8-b725-47dd-aa4c-4ea59928d878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:26.160255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-f7ddc1b8-b725-47dd-aa4c-4ea59928d878 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:26.161680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f7ddc1b8-b725-47dd-aa4c-4ea59928d878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:26.231612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad5826-3afc-4604-bb15-31b0be534d08 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:26.233410 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0fad5826-3afc-4604-bb15-31b0be534d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:26.247933 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1295/2543] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:49:26.250715 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfb64b-aeec-4b16-99c9-b6f35b7688cd None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:26.251004 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfb64b-aeec-4b16-99c9-b6f35b7688cd None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:26.251288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfb64b-aeec-4b16-99c9-b6f35b7688cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:26.252931 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad5826-3afc-4604-bb15-31b0be534d08 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:26.258205 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d51a9-aa32-4506-a55c-9e611cd696a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:26.260593 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d51a9-aa32-4506-a55c-9e611cd696a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:26.260987 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d51a9-aa32-4506-a55c-9e611cd696a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:26.261386 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d51a9-aa32-4506-a55c-9e611cd696a8 admin admin] PATH_INFO: `/v3/users/f516de59ae5a4adca5826185e5156d76` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:26.262748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d51a9-aa32-4506-a55c-9e611cd696a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f516de59ae5a4adca5826185e5156d76)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:26.264791 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d51a9-aa32-4506-a55c-9e611cd696a8 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:26.275254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-bbcfb64b-aeec-4b16-99c9-b6f35b7688cd None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:26.276630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-bbcfb64b-aeec-4b16-99c9-b6f35b7688cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:26.284827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1296/2544] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:26.288875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfb64b-aeec-4b16-99c9-b6f35b7688cd None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:26.298098 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfb64b-aeec-4b16-99c9-b6f35b7688cd None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:26.303329 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedf005-76bb-487e-9215-6ccf53e878aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:26.303989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedf005-76bb-487e-9215-6ccf53e878aa admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:26.304345 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedf005-76bb-487e-9215-6ccf53e878aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:26.304548 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedf005-76bb-487e-9215-6ccf53e878aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:26.306137 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eedf005-76bb-487e-9215-6ccf53e878aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:49:26.306137 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eedf005-76bb-487e-9215-6ccf53e878aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:26.307805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eedf005-76bb-487e-9215-6ccf53e878aa admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:26.320400 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1297/2545] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:49:26.326365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedf005-76bb-487e-9215-6ccf53e878aa admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:26.328860 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-397d51a9-aa32-4506-a55c-9e611cd696a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f516de59ae5a4adca5826185e5156d76'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:26.340324 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedf005-76bb-487e-9215-6ccf53e878aa admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:26.345110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19070ccf-cd6a-4bd5-8d30-6bd33973131f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:26.345463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-397d51a9-aa32-4506-a55c-9e611cd696a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f516de59ae5a4adca5826185e5156d76'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:26.345795 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19070ccf-cd6a-4bd5-8d30-6bd33973131f admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:26.346106 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19070ccf-cd6a-4bd5-8d30-6bd33973131f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:26.346379 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19070ccf-cd6a-4bd5-8d30-6bd33973131f admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:26.347063 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19070ccf-cd6a-4bd5-8d30-6bd33973131f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:26.348972 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19070ccf-cd6a-4bd5-8d30-6bd33973131f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:26.352947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-397d51a9-aa32-4506-a55c-9e611cd696a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f516de59ae5a4adca5826185e5156d76'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:26.354070 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-397d51a9-aa32-4506-a55c-9e611cd696a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f516de59ae5a4adca5826185e5156d76'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:26.362565 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-397d51a9-aa32-4506-a55c-9e611cd696a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f516de59ae5a4adca5826185e5156d76'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:26.369794 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1249/2546] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:26 2022] DELETE /identity/v3/users/f516de59ae5a4adca5826185e5156d76 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:26.374168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80122027-316c-4a09-8e9c-fd880e2c05ab None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:26.374459 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80122027-316c-4a09-8e9c-fd880e2c05ab None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:26.374744 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80122027-316c-4a09-8e9c-fd880e2c05ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:26.377225 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1298/2547] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:26 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:26.380828 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19070ccf-cd6a-4bd5-8d30-6bd33973131f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:26.382450 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-19070ccf-cd6a-4bd5-8d30-6bd33973131f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:26.393293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-80122027-316c-4a09-8e9c-fd880e2c05ab None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:26.394510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-80122027-316c-4a09-8e9c-fd880e2c05ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:26.398419 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:26.404190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:26.456826 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19070ccf-cd6a-4bd5-8d30-6bd33973131f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:26.463894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d89c57-f4d5-4dc5-a599-503833847c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:26.465863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d89c57-f4d5-4dc5-a599-503833847c9c admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:26.465863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d89c57-f4d5-4dc5-a599-503833847c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:26.465863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d89c57-f4d5-4dc5-a599-503833847c9c admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:26.466217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d89c57-f4d5-4dc5-a599-503833847c9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:26.469584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d89c57-f4d5-4dc5-a599-503833847c9c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:26.506325 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1250/2548] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:49:26.506325 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80122027-316c-4a09-8e9c-fd880e2c05ab None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:26.506926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-80122027-316c-4a09-8e9c-fd880e2c05ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:26.574435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1299/2549] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:26 2022] POST /identity/v3/users => generated 282 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:49:26.578058 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d89c57-f4d5-4dc5-a599-503833847c9c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:26.591444 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d89c57-f4d5-4dc5-a599-503833847c9c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:26.596444 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d58fd8-ac52-4b4c-946e-ee7ff8103b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:26.597146 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d58fd8-ac52-4b4c-946e-ee7ff8103b95 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:26.597699 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d58fd8-ac52-4b4c-946e-ee7ff8103b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:26.598016 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d58fd8-ac52-4b4c-946e-ee7ff8103b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:26.598752 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d58fd8-ac52-4b4c-946e-ee7ff8103b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:26.600389 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d58fd8-ac52-4b4c-946e-ee7ff8103b95 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:26.608594 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80122027-316c-4a09-8e9c-fd880e2c05ab None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:26.615073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1300/2550] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:49:26.615542 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66159d-44c5-48ab-ae64-ae23bbc00ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:26.615542 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66159d-44c5-48ab-ae64-ae23bbc00ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:26.615542 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66159d-44c5-48ab-ae64-ae23bbc00ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:26.615542 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66159d-44c5-48ab-ae64-ae23bbc00ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:26.616159 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d58fd8-ac52-4b4c-946e-ee7ff8103b95 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:26.625447 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66159d-44c5-48ab-ae64-ae23bbc00ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:26.627322 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66159d-44c5-48ab-ae64-ae23bbc00ed8 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:26.627529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d58fd8-ac52-4b4c-946e-ee7ff8103b95 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:26.632190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f71a4-234b-48d3-bde6-c282b16f012d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:26.632834 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f71a4-234b-48d3-bde6-c282b16f012d admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:26.633122 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f71a4-234b-48d3-bde6-c282b16f012d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:26.633451 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f71a4-234b-48d3-bde6-c282b16f012d admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:26.634223 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f71a4-234b-48d3-bde6-c282b16f012d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:26.635826 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f71a4-234b-48d3-bde6-c282b16f012d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:26.636888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1251/2551] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:49:26.646381 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1301/2552] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:49:26.650412 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66159d-44c5-48ab-ae64-ae23bbc00ed8 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:26.660230 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66159d-44c5-48ab-ae64-ae23bbc00ed8 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:26.666776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864bca89-17a9-4e3d-89e8-09be0c1114bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:26.667696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bca89-17a9-4e3d-89e8-09be0c1114bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:26.667993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bca89-17a9-4e3d-89e8-09be0c1114bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:26.668247 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bca89-17a9-4e3d-89e8-09be0c1114bf admin admin] PATH_INFO: `/v3/projects/d7cc83ee86374c2089dfe52ece773854/users/f4b1f57ef6494b98ac48b8b402001161/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:26.685551 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864bca89-17a9-4e3d-89e8-09be0c1114bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7cc83ee86374c2089dfe52ece773854, user_id=f4b1f57ef6494b98ac48b8b402001161, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:26.688421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864bca89-17a9-4e3d-89e8-09be0c1114bf admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:26.703155 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1252/2553] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:26 2022] PUT /identity/v3/projects/d7cc83ee86374c2089dfe52ece773854/users/f4b1f57ef6494b98ac48b8b402001161/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:26.716025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39790420-5ebc-4745-85d5-dd2b88c913ae None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:26.716808 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39790420-5ebc-4745-85d5-dd2b88c913ae None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:26.717528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39790420-5ebc-4745-85d5-dd2b88c913ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:26.747628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-39790420-5ebc-4745-85d5-dd2b88c913ae None None] MFA Rules not processed for user `f4b1f57ef6494b98ac48b8b402001161`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:26.748894 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-39790420-5ebc-4745-85d5-dd2b88c913ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:26.818795 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1302/2554] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:26 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:26.823197 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864bca89-17a9-4e3d-89e8-09be0c1114bf admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:26.900952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864bca89-17a9-4e3d-89e8-09be0c1114bf admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:26.911763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e045b3-6dd3-4003-b3d7-475f5226def2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:26.912787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e045b3-6dd3-4003-b3d7-475f5226def2 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:26.913359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e045b3-6dd3-4003-b3d7-475f5226def2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:26.913794 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e045b3-6dd3-4003-b3d7-475f5226def2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:26.915315 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e045b3-6dd3-4003-b3d7-475f5226def2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:26.917653 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e045b3-6dd3-4003-b3d7-475f5226def2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:26.935160 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39790420-5ebc-4745-85d5-dd2b88c913ae None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:26.936727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-39790420-5ebc-4745-85d5-dd2b88c913ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:26.951547 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39790420-5ebc-4745-85d5-dd2b88c913ae None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:26.959376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1253/2555] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:26 2022] POST /identity/v3/projects => generated 363 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:26.963858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e045b3-6dd3-4003-b3d7-475f5226def2 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:26.980621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:27.018593 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa763e5-4422-4708-b7c1-b72863624264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:27.019892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa763e5-4422-4708-b7c1-b72863624264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:27.020283 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa763e5-4422-4708-b7c1-b72863624264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:27.020688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa763e5-4422-4708-b7c1-b72863624264 admin admin] PATH_INFO: `/v3/projects/0b81af8c96e64846a84740d38513965f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:27.022125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa763e5-4422-4708-b7c1-b72863624264 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b81af8c96e64846a84740d38513965f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:27.024880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa763e5-4422-4708-b7c1-b72863624264 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:27.041936 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e045b3-6dd3-4003-b3d7-475f5226def2 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:27.047779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6267af8-3306-49ad-9f0c-b072e263424c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:27.048413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6267af8-3306-49ad-9f0c-b072e263424c admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:27.048698 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6267af8-3306-49ad-9f0c-b072e263424c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:27.048974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6267af8-3306-49ad-9f0c-b072e263424c admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:27.049670 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6267af8-3306-49ad-9f0c-b072e263424c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:27.050196 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:27.052333 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6267af8-3306-49ad-9f0c-b072e263424c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:27.079851 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7fa763e5-4422-4708-b7c1-b72863624264 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b81af8c96e64846a84740d38513965f', 'request_id': 'req-7fa763e5-4422-4708-b7c1-b72863624264'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:27.088960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7fa763e5-4422-4708-b7c1-b72863624264 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b81af8c96e64846a84740d38513965f', 'request_id': 'req-7fa763e5-4422-4708-b7c1-b72863624264'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:27.092039 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1254/2556] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:26 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:49:27.092842 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6267af8-3306-49ad-9f0c-b072e263424c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:27.098068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7fa763e5-4422-4708-b7c1-b72863624264 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b81af8c96e64846a84740d38513965f', 'request_id': 'req-7fa763e5-4422-4708-b7c1-b72863624264'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:27.105168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6267af8-3306-49ad-9f0c-b072e263424c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:27.111451 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7fa763e5-4422-4708-b7c1-b72863624264 admin admin] The token cache is being invalidate because project 0b81af8c96e64846a84740d38513965f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:27.111756 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7fa763e5-4422-4708-b7c1-b72863624264 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:27.112472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7fa763e5-4422-4708-b7c1-b72863624264 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:27.114063 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1303/2557] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:26 2022] DELETE /identity/v3/projects/0b81af8c96e64846a84740d38513965f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:27.118394 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa763e5-4422-4708-b7c1-b72863624264 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:27.120244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7fa763e5-4422-4708-b7c1-b72863624264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:27.181656 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e41dad-2d66-477c-8093-94c818c2725b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:27.182618 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e41dad-2d66-477c-8093-94c818c2725b admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:27.182938 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e41dad-2d66-477c-8093-94c818c2725b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:27.183276 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e41dad-2d66-477c-8093-94c818c2725b admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:27.184898 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e41dad-2d66-477c-8093-94c818c2725b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:27.188281 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e41dad-2d66-477c-8093-94c818c2725b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:27.196710 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa763e5-4422-4708-b7c1-b72863624264 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:27.199061 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1255/2558] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:27.202377 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e41dad-2d66-477c-8093-94c818c2725b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:27.205536 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514ef7e-5256-4fdc-be98-6c8c7b6bac74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:27.206182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-72e41dad-2d66-477c-8093-94c818c2725b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:27.206634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514ef7e-5256-4fdc-be98-6c8c7b6bac74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:27.206987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514ef7e-5256-4fdc-be98-6c8c7b6bac74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:27.207555 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514ef7e-5256-4fdc-be98-6c8c7b6bac74 admin admin] PATH_INFO: `/v3/users/a7da6dbe8d06480281cdc07d5d987237` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:27.209210 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4514ef7e-5256-4fdc-be98-6c8c7b6bac74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7da6dbe8d06480281cdc07d5d987237)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:27.211273 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4514ef7e-5256-4fdc-be98-6c8c7b6bac74 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:27.238413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e41dad-2d66-477c-8093-94c818c2725b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:27.247954 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765e579-45c9-4a2b-a608-feebc84609cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:27.248857 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765e579-45c9-4a2b-a608-feebc84609cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:27.249168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765e579-45c9-4a2b-a608-feebc84609cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:27.249416 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765e579-45c9-4a2b-a608-feebc84609cf admin admin] PATH_INFO: `/v3/projects/e61d5589fc11421bb0c7eef9dce7e272/users/fa8a3f95411f421da12bbed1af147395/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:27.270130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765e579-45c9-4a2b-a608-feebc84609cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e61d5589fc11421bb0c7eef9dce7e272, user_id=fa8a3f95411f421da12bbed1af147395, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:27.272993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765e579-45c9-4a2b-a608-feebc84609cf admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:27.273379 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4514ef7e-5256-4fdc-be98-6c8c7b6bac74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7da6dbe8d06480281cdc07d5d987237'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:27.274991 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4514ef7e-5256-4fdc-be98-6c8c7b6bac74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7da6dbe8d06480281cdc07d5d987237'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:27.275664 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4514ef7e-5256-4fdc-be98-6c8c7b6bac74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7da6dbe8d06480281cdc07d5d987237'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:27.317148 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4514ef7e-5256-4fdc-be98-6c8c7b6bac74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7da6dbe8d06480281cdc07d5d987237'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:27.320564 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1256/2559] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:27 2022] PUT /identity/v3/projects/e61d5589fc11421bb0c7eef9dce7e272/users/fa8a3f95411f421da12bbed1af147395/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:27.324750 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765e579-45c9-4a2b-a608-feebc84609cf admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:27.328193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2765e579-45c9-4a2b-a608-feebc84609cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:27.328333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4514ef7e-5256-4fdc-be98-6c8c7b6bac74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7da6dbe8d06480281cdc07d5d987237'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:27.346233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1304/2560] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:27 2022] DELETE /identity/v3/users/a7da6dbe8d06480281cdc07d5d987237 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:27.409093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765e579-45c9-4a2b-a608-feebc84609cf admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:27.440656 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:27.499772 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b1491a-085b-48dc-b49f-71677a2a3f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:27.500775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1491a-085b-48dc-b49f-71677a2a3f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:27.501066 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1491a-085b-48dc-b49f-71677a2a3f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:27.501900 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1491a-085b-48dc-b49f-71677a2a3f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:27.503494 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b1491a-085b-48dc-b49f-71677a2a3f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:27.506889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b1491a-085b-48dc-b49f-71677a2a3f87 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:27.522559 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1257/2561] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:49:27.527618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514ef7e-5256-4fdc-be98-6c8c7b6bac74 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:27.540604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514ef7e-5256-4fdc-be98-6c8c7b6bac74 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:27.545713 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc9813-9b4e-47ae-b8e4-7b04bcd69c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:27.546756 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9813-9b4e-47ae-b8e4-7b04bcd69c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:27.547106 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9813-9b4e-47ae-b8e4-7b04bcd69c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:27.547456 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9813-9b4e-47ae-b8e4-7b04bcd69c08 admin admin] PATH_INFO: `/v3/projects/e61d5589fc11421bb0c7eef9dce7e272/users/fa8a3f95411f421da12bbed1af147395/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:27.549523 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc9813-9b4e-47ae-b8e4-7b04bcd69c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e61d5589fc11421bb0c7eef9dce7e272, user_id=fa8a3f95411f421da12bbed1af147395, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:27.552270 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc9813-9b4e-47ae-b8e4-7b04bcd69c08 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:27.564285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1305/2562] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:27 2022] PUT /identity/v3/projects/e61d5589fc11421bb0c7eef9dce7e272/users/fa8a3f95411f421da12bbed1af147395/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:27.574900 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6148a-9074-498b-aaf8-84752a5a48ee None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:27.575277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6148a-9074-498b-aaf8-84752a5a48ee None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:27.575630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6148a-9074-498b-aaf8-84752a5a48ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:27.609160 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-43a6148a-9074-498b-aaf8-84752a5a48ee None None] MFA Rules not processed for user `fa8a3f95411f421da12bbed1af147395`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:27.610708 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-43a6148a-9074-498b-aaf8-84752a5a48ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:27.693326 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1258/2563] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:27 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:49:27.706811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc9813-9b4e-47ae-b8e4-7b04bcd69c08 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:27.708558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-bbbc9813-9b4e-47ae-b8e4-7b04bcd69c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:27.719388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6148a-9074-498b-aaf8-84752a5a48ee None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:27.723461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-43a6148a-9074-498b-aaf8-84752a5a48ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:27.725342 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:27.779002 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc9813-9b4e-47ae-b8e4-7b04bcd69c08 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:27.782709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf64b5-be93-4764-961e-a3f56c63576c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:27.783357 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf64b5-be93-4764-961e-a3f56c63576c service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:27.783643 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf64b5-be93-4764-961e-a3f56c63576c service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:27.783920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf64b5-be93-4764-961e-a3f56c63576c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:27.793323 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf64b5-be93-4764-961e-a3f56c63576c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:27.794807 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf64b5-be93-4764-961e-a3f56c63576c service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:27.804728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1306/2564] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:27 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:27.810374 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6148a-9074-498b-aaf8-84752a5a48ee None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:27.816830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7d67a-3a3a-4442-abbe-45934a7ab762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:27.817643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7d67a-3a3a-4442-abbe-45934a7ab762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:27.817926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7d67a-3a3a-4442-abbe-45934a7ab762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:27.818234 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7d67a-3a3a-4442-abbe-45934a7ab762 admin admin] PATH_INFO: `/v3/projects/16d10495cd9d4736a7410e4e69a9fb86` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:27.819446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7d67a-3a3a-4442-abbe-45934a7ab762 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16d10495cd9d4736a7410e4e69a9fb86)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:27.822024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7d67a-3a3a-4442-abbe-45934a7ab762 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:27.876685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:27.879990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf64b5-be93-4764-961e-a3f56c63576c service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:27.880439 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8cf7d67a-3a3a-4442-abbe-45934a7ab762 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16d10495cd9d4736a7410e4e69a9fb86', 'request_id': 'req-8cf7d67a-3a3a-4442-abbe-45934a7ab762'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:27.888075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8cf7d67a-3a3a-4442-abbe-45934a7ab762 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16d10495cd9d4736a7410e4e69a9fb86', 'request_id': 'req-8cf7d67a-3a3a-4442-abbe-45934a7ab762'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:27.889718 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf64b5-be93-4764-961e-a3f56c63576c service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:27.895572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8cf7d67a-3a3a-4442-abbe-45934a7ab762 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16d10495cd9d4736a7410e4e69a9fb86', 'request_id': 'req-8cf7d67a-3a3a-4442-abbe-45934a7ab762'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:27.907257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8cf7d67a-3a3a-4442-abbe-45934a7ab762 admin admin] The token cache is being invalidate because project 16d10495cd9d4736a7410e4e69a9fb86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:27.907623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8cf7d67a-3a3a-4442-abbe-45934a7ab762 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:27.908348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8cf7d67a-3a3a-4442-abbe-45934a7ab762 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:27.910044 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1259/2565] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:27 2022] DELETE /identity/v3/projects/16d10495cd9d4736a7410e4e69a9fb86 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:27.914695 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb12b96-2043-48e3-8f3a-60d8269a62b2 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:27.915406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb12b96-2043-48e3-8f3a-60d8269a62b2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:27.915725 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb12b96-2043-48e3-8f3a-60d8269a62b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:27.934138 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0eb12b96-2043-48e3-8f3a-60d8269a62b2 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:27.935366 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0eb12b96-2043-48e3-8f3a-60d8269a62b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:27.939940 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c0d15-b08b-4fef-90e0-8deb35e9977e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:27.940653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c0d15-b08b-4fef-90e0-8deb35e9977e service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:27.940948 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c0d15-b08b-4fef-90e0-8deb35e9977e service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:27.941227 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c0d15-b08b-4fef-90e0-8deb35e9977e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:27.943111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-681c0d15-b08b-4fef-90e0-8deb35e9977e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:28.000962 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1260/2566] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:49:28.007759 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c0d15-b08b-4fef-90e0-8deb35e9977e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:28.009348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4facd-5073-4a81-aa9e-8baede3d619b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:28.009636 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4facd-5073-4a81-aa9e-8baede3d619b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:28.009776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c0d15-b08b-4fef-90e0-8deb35e9977e service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:28.009911 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4facd-5073-4a81-aa9e-8baede3d619b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:28.020006 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1307/2567] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:27 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:49:28.029290 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d3b4facd-5073-4a81-aa9e-8baede3d619b None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:28.030539 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d3b4facd-5073-4a81-aa9e-8baede3d619b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:28.038773 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1261/2568] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:28.043571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c0d15-b08b-4fef-90e0-8deb35e9977e service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:28.054193 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c0d15-b08b-4fef-90e0-8deb35e9977e service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:28.058880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088907f1-a6ea-457e-8fb4-d5b749dcadfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:28.059529 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088907f1-a6ea-457e-8fb4-d5b749dcadfb admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:28.059818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088907f1-a6ea-457e-8fb4-d5b749dcadfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:28.060105 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088907f1-a6ea-457e-8fb4-d5b749dcadfb admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:28.060860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088907f1-a6ea-457e-8fb4-d5b749dcadfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:49:28.061169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088907f1-a6ea-457e-8fb4-d5b749dcadfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:28.062823 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088907f1-a6ea-457e-8fb4-d5b749dcadfb admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:28.076948 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1308/2569] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:49:28.081442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4facd-5073-4a81-aa9e-8baede3d619b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:28.091976 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4facd-5073-4a81-aa9e-8baede3d619b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:28.097722 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca284f31-f379-4d9d-86bf-91651301f326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:28.098879 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca284f31-f379-4d9d-86bf-91651301f326 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:28.099171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca284f31-f379-4d9d-86bf-91651301f326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:28.099474 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca284f31-f379-4d9d-86bf-91651301f326 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:28.100226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca284f31-f379-4d9d-86bf-91651301f326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:28.102102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca284f31-f379-4d9d-86bf-91651301f326 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:28.139776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1262/2570] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:28 2022] POST /identity/v3/projects => generated 393 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:28.141379 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088907f1-a6ea-457e-8fb4-d5b749dcadfb admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:28.155220 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088907f1-a6ea-457e-8fb4-d5b749dcadfb admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:28.207342 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:28.214498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2456e1-1c9d-4c21-a1e3-ec45c62cd55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:28.215168 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2456e1-1c9d-4c21-a1e3-ec45c62cd55b admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:28.215452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2456e1-1c9d-4c21-a1e3-ec45c62cd55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:28.215726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2456e1-1c9d-4c21-a1e3-ec45c62cd55b admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:28.216410 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2456e1-1c9d-4c21-a1e3-ec45c62cd55b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:28.218800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2456e1-1c9d-4c21-a1e3-ec45c62cd55b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:28.237740 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1309/2571] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:28 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:28.241837 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca284f31-f379-4d9d-86bf-91651301f326 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:28.252238 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca284f31-f379-4d9d-86bf-91651301f326 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:28.256867 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d217ea-ff76-4e8c-a2f2-7d477ff1a4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:28.257509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d217ea-ff76-4e8c-a2f2-7d477ff1a4b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:28.257789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d217ea-ff76-4e8c-a2f2-7d477ff1a4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:28.258088 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d217ea-ff76-4e8c-a2f2-7d477ff1a4b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:28.259406 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d217ea-ff76-4e8c-a2f2-7d477ff1a4b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:28.260985 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d217ea-ff76-4e8c-a2f2-7d477ff1a4b2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:28.271088 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1263/2572] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:28.275688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2456e1-1c9d-4c21-a1e3-ec45c62cd55b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:28.285817 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2456e1-1c9d-4c21-a1e3-ec45c62cd55b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:28.291128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07407df3-2c67-47ee-82d7-0646c19848a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:28.291812 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07407df3-2c67-47ee-82d7-0646c19848a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:28.292097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07407df3-2c67-47ee-82d7-0646c19848a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:28.292376 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07407df3-2c67-47ee-82d7-0646c19848a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:28.293097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07407df3-2c67-47ee-82d7-0646c19848a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:28.294755 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07407df3-2c67-47ee-82d7-0646c19848a0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:28.304463 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1310/2573] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:28.308601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d217ea-ff76-4e8c-a2f2-7d477ff1a4b2 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:28.318975 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d217ea-ff76-4e8c-a2f2-7d477ff1a4b2 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:28.324305 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317faa0-745a-4253-8733-f075786034f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:28.325427 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317faa0-745a-4253-8733-f075786034f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:28.325711 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317faa0-745a-4253-8733-f075786034f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:28.326012 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317faa0-745a-4253-8733-f075786034f0 admin admin] PATH_INFO: `/v3/projects/3b34f9712a0041d5b81dc10d0e7a3d4e/users/3e60309c4d374f42ad8e83831452591b/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:28.341232 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317faa0-745a-4253-8733-f075786034f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b34f9712a0041d5b81dc10d0e7a3d4e, user_id=3e60309c4d374f42ad8e83831452591b, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:28.343926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317faa0-745a-4253-8733-f075786034f0 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:28.356762 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1264/2574] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:28 2022] PUT /identity/v3/projects/3b34f9712a0041d5b81dc10d0e7a3d4e/users/3e60309c4d374f42ad8e83831452591b/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:28.366923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d9109-9cdd-4978-b919-ddcc20169cf1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:28.367237 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d9109-9cdd-4978-b919-ddcc20169cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:28.367517 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d9109-9cdd-4978-b919-ddcc20169cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:28.393793 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-652d9109-9cdd-4978-b919-ddcc20169cf1 None None] MFA Rules not processed for user `3e60309c4d374f42ad8e83831452591b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:28.395025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-652d9109-9cdd-4978-b919-ddcc20169cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:28.460987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1311/2575] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:28.466103 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317faa0-745a-4253-8733-f075786034f0 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:28.529376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317faa0-745a-4253-8733-f075786034f0 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:28.534168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cac692-c1ea-47cd-8c25-9bc1f7859571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:28.534827 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cac692-c1ea-47cd-8c25-9bc1f7859571 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:28.535113 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cac692-c1ea-47cd-8c25-9bc1f7859571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:28.535390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cac692-c1ea-47cd-8c25-9bc1f7859571 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:28.536090 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cac692-c1ea-47cd-8c25-9bc1f7859571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:28.537997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cac692-c1ea-47cd-8c25-9bc1f7859571 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:28.551993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:28.569908 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1265/2576] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:28 2022] POST /identity/v3/projects => generated 391 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:28.571673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d9109-9cdd-4978-b919-ddcc20169cf1 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:28.582733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d9109-9cdd-4978-b919-ddcc20169cf1 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:28.640594 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e58d7b-e2a9-4c4a-a74a-dea04ccd95d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:28.641275 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e58d7b-e2a9-4c4a-a74a-dea04ccd95d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:28.641597 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e58d7b-e2a9-4c4a-a74a-dea04ccd95d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:28.641881 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e58d7b-e2a9-4c4a-a74a-dea04ccd95d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:28.642625 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e58d7b-e2a9-4c4a-a74a-dea04ccd95d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:28.644586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e58d7b-e2a9-4c4a-a74a-dea04ccd95d4 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:28.670133 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1312/2577] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:28 2022] POST /identity/v3/users => generated 302 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:28.671381 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cac692-c1ea-47cd-8c25-9bc1f7859571 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:28.683461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cac692-c1ea-47cd-8c25-9bc1f7859571 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:28.688644 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea00f3-3d0f-4ebe-b1b7-52f93861525b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:28.689324 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea00f3-3d0f-4ebe-b1b7-52f93861525b admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:28.689642 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea00f3-3d0f-4ebe-b1b7-52f93861525b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:28.689925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea00f3-3d0f-4ebe-b1b7-52f93861525b admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:28.690973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea00f3-3d0f-4ebe-b1b7-52f93861525b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:28.692420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea00f3-3d0f-4ebe-b1b7-52f93861525b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:28.705125 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1266/2578] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:49:28.709080 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e58d7b-e2a9-4c4a-a74a-dea04ccd95d4 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:28.719617 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e58d7b-e2a9-4c4a-a74a-dea04ccd95d4 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:28.725114 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadac59b-501a-44a0-a23d-5b8462cf3972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:28.726050 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadac59b-501a-44a0-a23d-5b8462cf3972 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:28.726399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadac59b-501a-44a0-a23d-5b8462cf3972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:28.726685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadac59b-501a-44a0-a23d-5b8462cf3972 admin admin] PATH_INFO: `/v3/projects/7b0f9ea2f49a4cac958fc868e48011b7/users/386a76566f864e358a5fc0d2f6f0bda7/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:28.741669 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadac59b-501a-44a0-a23d-5b8462cf3972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0f9ea2f49a4cac958fc868e48011b7, user_id=386a76566f864e358a5fc0d2f6f0bda7, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:28.744889 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadac59b-501a-44a0-a23d-5b8462cf3972 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:28.758103 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1313/2579] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:28 2022] PUT /identity/v3/projects/7b0f9ea2f49a4cac958fc868e48011b7/users/386a76566f864e358a5fc0d2f6f0bda7/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:28.762538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea00f3-3d0f-4ebe-b1b7-52f93861525b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:28.772477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea00f3-3d0f-4ebe-b1b7-52f93861525b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:28.827615 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d89899-94ac-4811-9d6d-f07f077b44a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:28.828315 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d89899-94ac-4811-9d6d-f07f077b44a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:28.828663 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d89899-94ac-4811-9d6d-f07f077b44a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:28.829106 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d89899-94ac-4811-9d6d-f07f077b44a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:28.829904 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d89899-94ac-4811-9d6d-f07f077b44a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:28.831537 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d89899-94ac-4811-9d6d-f07f077b44a6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:28.841274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1267/2580] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:49:28.845494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadac59b-501a-44a0-a23d-5b8462cf3972 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:28.857190 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadac59b-501a-44a0-a23d-5b8462cf3972 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:28.861901 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d475a16-b82b-42ec-83a9-5a15c42e27fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:28.862854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d475a16-b82b-42ec-83a9-5a15c42e27fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:28.863139 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d475a16-b82b-42ec-83a9-5a15c42e27fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:28.863418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d475a16-b82b-42ec-83a9-5a15c42e27fe admin admin] PATH_INFO: `/v3/projects/7b0f9ea2f49a4cac958fc868e48011b7/users/386a76566f864e358a5fc0d2f6f0bda7/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:28.865383 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d475a16-b82b-42ec-83a9-5a15c42e27fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0f9ea2f49a4cac958fc868e48011b7, user_id=386a76566f864e358a5fc0d2f6f0bda7, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:28.868707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d475a16-b82b-42ec-83a9-5a15c42e27fe admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:28.879332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1314/2581] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:28 2022] PUT /identity/v3/projects/7b0f9ea2f49a4cac958fc868e48011b7/users/386a76566f864e358a5fc0d2f6f0bda7/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:28.890141 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219ee6e-ad6f-4810-b1cb-122712435849 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:28.890437 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219ee6e-ad6f-4810-b1cb-122712435849 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:28.890721 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219ee6e-ad6f-4810-b1cb-122712435849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:28.917536 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a219ee6e-ad6f-4810-b1cb-122712435849 None None] MFA Rules not processed for user `386a76566f864e358a5fc0d2f6f0bda7`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:28.919248 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a219ee6e-ad6f-4810-b1cb-122712435849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:28.991111 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1268/2582] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:29.003153 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d475a16-b82b-42ec-83a9-5a15c42e27fe admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:29.005170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2d475a16-b82b-42ec-83a9-5a15c42e27fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:29.027522 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:29.069300 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d475a16-b82b-42ec-83a9-5a15c42e27fe admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:29.073455 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a2d7f-93c1-4af7-84d8-50228638b68f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:29.074355 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a2d7f-93c1-4af7-84d8-50228638b68f service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:29.074707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a2d7f-93c1-4af7-84d8-50228638b68f service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:29.074982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a2d7f-93c1-4af7-84d8-50228638b68f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:29.084764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a2d7f-93c1-4af7-84d8-50228638b68f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:29.086288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a2d7f-93c1-4af7-84d8-50228638b68f service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:29.096064 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1315/2583] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:49:29.146685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219ee6e-ad6f-4810-b1cb-122712435849 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:29.148130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a219ee6e-ad6f-4810-b1cb-122712435849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:29.161474 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219ee6e-ad6f-4810-b1cb-122712435849 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:29.166159 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2de544-bd19-48bc-9975-0a04410dc96e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:29.166947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2de544-bd19-48bc-9975-0a04410dc96e service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:29.167322 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2de544-bd19-48bc-9975-0a04410dc96e service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:29.167623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2de544-bd19-48bc-9975-0a04410dc96e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:29.233362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2de544-bd19-48bc-9975-0a04410dc96e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:29.234867 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2de544-bd19-48bc-9975-0a04410dc96e service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:29.244758 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1269/2584] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:30.793993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a2d7f-93c1-4af7-84d8-50228638b68f service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:30.797074 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-bd4a2d7f-93c1-4af7-84d8-50228638b68f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:30.872017 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a2d7f-93c1-4af7-84d8-50228638b68f service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:30.876628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebcf87-64af-47ce-ac48-5ecb32974627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:30.877754 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebcf87-64af-47ce-ac48-5ecb32974627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:30.878065 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebcf87-64af-47ce-ac48-5ecb32974627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:30.878341 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebcf87-64af-47ce-ac48-5ecb32974627 admin admin] PATH_INFO: `/v3/users/f4b1f57ef6494b98ac48b8b402001161` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:30.879543 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cebcf87-64af-47ce-ac48-5ecb32974627 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4b1f57ef6494b98ac48b8b402001161)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:30.881453 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cebcf87-64af-47ce-ac48-5ecb32974627 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:30.927841 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9cebcf87-64af-47ce-ac48-5ecb32974627 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4b1f57ef6494b98ac48b8b402001161'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:30.928506 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9cebcf87-64af-47ce-ac48-5ecb32974627 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4b1f57ef6494b98ac48b8b402001161'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:30.929115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9cebcf87-64af-47ce-ac48-5ecb32974627 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4b1f57ef6494b98ac48b8b402001161'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:30.941477 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9cebcf87-64af-47ce-ac48-5ecb32974627 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4b1f57ef6494b98ac48b8b402001161'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:30.949230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9cebcf87-64af-47ce-ac48-5ecb32974627 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4b1f57ef6494b98ac48b8b402001161'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:30.963739 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1316/2585] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:30 2022] DELETE /identity/v3/users/f4b1f57ef6494b98ac48b8b402001161 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:30.968215 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90735abb-b261-46e0-9d1b-3220c0e52ee9 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:30.968711 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90735abb-b261-46e0-9d1b-3220c0e52ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:30.968999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90735abb-b261-46e0-9d1b-3220c0e52ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:30.976384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:30.988962 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-90735abb-b261-46e0-9d1b-3220c0e52ee9 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:30.990216 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-90735abb-b261-46e0-9d1b-3220c0e52ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:31.054875 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1270/2586] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:31.064835 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebcf87-64af-47ce-ac48-5ecb32974627 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:31.066341 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9cebcf87-64af-47ce-ac48-5ecb32974627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:31.117380 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:31.136016 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebcf87-64af-47ce-ac48-5ecb32974627 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:31.139787 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b9246-5661-49f7-9c34-c809629b96fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:31.140628 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b9246-5661-49f7-9c34-c809629b96fe service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:31.140989 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b9246-5661-49f7-9c34-c809629b96fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:31.141346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b9246-5661-49f7-9c34-c809629b96fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:31.151369 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b9246-5661-49f7-9c34-c809629b96fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:31.152826 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b9246-5661-49f7-9c34-c809629b96fe service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:31.162632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1317/2587] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:31.460645 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90735abb-b261-46e0-9d1b-3220c0e52ee9 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:31.462927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-90735abb-b261-46e0-9d1b-3220c0e52ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:31.483421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90735abb-b261-46e0-9d1b-3220c0e52ee9 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:31.488029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4530e-ca6c-4d11-9a04-2b702292f334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:31.488893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4530e-ca6c-4d11-9a04-2b702292f334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:31.489176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4530e-ca6c-4d11-9a04-2b702292f334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:31.489451 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4530e-ca6c-4d11-9a04-2b702292f334 admin admin] PATH_INFO: `/v3/projects/d7cc83ee86374c2089dfe52ece773854` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:31.490826 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b4530e-ca6c-4d11-9a04-2b702292f334 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7cc83ee86374c2089dfe52ece773854)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:31.493353 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b4530e-ca6c-4d11-9a04-2b702292f334 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:31.542137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-13b4530e-ca6c-4d11-9a04-2b702292f334 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7cc83ee86374c2089dfe52ece773854', 'request_id': 'req-13b4530e-ca6c-4d11-9a04-2b702292f334'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:31.547708 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-13b4530e-ca6c-4d11-9a04-2b702292f334 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7cc83ee86374c2089dfe52ece773854', 'request_id': 'req-13b4530e-ca6c-4d11-9a04-2b702292f334'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:31.554973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-13b4530e-ca6c-4d11-9a04-2b702292f334 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7cc83ee86374c2089dfe52ece773854', 'request_id': 'req-13b4530e-ca6c-4d11-9a04-2b702292f334'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:31.566337 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-13b4530e-ca6c-4d11-9a04-2b702292f334 admin admin] The token cache is being invalidate because project d7cc83ee86374c2089dfe52ece773854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:31.566619 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-13b4530e-ca6c-4d11-9a04-2b702292f334 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:31.567237 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-13b4530e-ca6c-4d11-9a04-2b702292f334 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:31.568775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1271/2588] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:31 2022] DELETE /identity/v3/projects/d7cc83ee86374c2089dfe52ece773854 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:31.573317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b9246-5661-49f7-9c34-c809629b96fe service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:31.582845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b9246-5661-49f7-9c34-c809629b96fe service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:31.640587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cca405-4814-4e4d-957b-7f9e4eab9694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:31.641468 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cca405-4814-4e4d-957b-7f9e4eab9694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:31.641757 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cca405-4814-4e4d-957b-7f9e4eab9694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:31.642068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cca405-4814-4e4d-957b-7f9e4eab9694 admin admin] PATH_INFO: `/v3/users/fa8a3f95411f421da12bbed1af147395` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:31.643314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cca405-4814-4e4d-957b-7f9e4eab9694 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa8a3f95411f421da12bbed1af147395)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:31.645248 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cca405-4814-4e4d-957b-7f9e4eab9694 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:31.693336 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-36cca405-4814-4e4d-957b-7f9e4eab9694 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa8a3f95411f421da12bbed1af147395'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:31.694136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-36cca405-4814-4e4d-957b-7f9e4eab9694 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa8a3f95411f421da12bbed1af147395'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:31.694805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-36cca405-4814-4e4d-957b-7f9e4eab9694 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa8a3f95411f421da12bbed1af147395'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:31.709986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-36cca405-4814-4e4d-957b-7f9e4eab9694 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa8a3f95411f421da12bbed1af147395'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:31.723362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-36cca405-4814-4e4d-957b-7f9e4eab9694 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa8a3f95411f421da12bbed1af147395'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:31.739274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1318/2589] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:31 2022] DELETE /identity/v3/users/fa8a3f95411f421da12bbed1af147395 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:32.046312 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4530e-ca6c-4d11-9a04-2b702292f334 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:32.048109 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-13b4530e-ca6c-4d11-9a04-2b702292f334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:32.072979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4530e-ca6c-4d11-9a04-2b702292f334 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:32.136647 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f7937-acaa-4485-9fdd-206863941f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:32.137527 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f7937-acaa-4485-9fdd-206863941f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:32.137813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f7937-acaa-4485-9fdd-206863941f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:32.138126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f7937-acaa-4485-9fdd-206863941f4d admin admin] PATH_INFO: `/v3/projects/e61d5589fc11421bb0c7eef9dce7e272` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:32.139328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f7937-acaa-4485-9fdd-206863941f4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e61d5589fc11421bb0c7eef9dce7e272)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:32.141251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f7937-acaa-4485-9fdd-206863941f4d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:32.176298 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:32.193237 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dc2f7937-acaa-4485-9fdd-206863941f4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e61d5589fc11421bb0c7eef9dce7e272', 'request_id': 'req-dc2f7937-acaa-4485-9fdd-206863941f4d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:32.202379 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dc2f7937-acaa-4485-9fdd-206863941f4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e61d5589fc11421bb0c7eef9dce7e272', 'request_id': 'req-dc2f7937-acaa-4485-9fdd-206863941f4d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:32.209866 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dc2f7937-acaa-4485-9fdd-206863941f4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e61d5589fc11421bb0c7eef9dce7e272', 'request_id': 'req-dc2f7937-acaa-4485-9fdd-206863941f4d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:32.223684 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dc2f7937-acaa-4485-9fdd-206863941f4d admin admin] The token cache is being invalidate because project e61d5589fc11421bb0c7eef9dce7e272 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:32.223975 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dc2f7937-acaa-4485-9fdd-206863941f4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:32.224620 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dc2f7937-acaa-4485-9fdd-206863941f4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:32.230089 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1272/2590] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:32 2022] DELETE /identity/v3/projects/e61d5589fc11421bb0c7eef9dce7e272 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:32.231919 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ac482-e687-4ef3-822d-663b42f044c8 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:32.232392 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ac482-e687-4ef3-822d-663b42f044c8 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:32.232610 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ac482-e687-4ef3-822d-663b42f044c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:32.252663 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5f9ac482-e687-4ef3-822d-663b42f044c8 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:32.253877 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5f9ac482-e687-4ef3-822d-663b42f044c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:32.325596 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1319/2591] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:32.334580 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcff10-7fc9-4102-b985-111ca4037344 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:32.334868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcff10-7fc9-4102-b985-111ca4037344 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:32.335242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcff10-7fc9-4102-b985-111ca4037344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:32.353144 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-72bcff10-7fc9-4102-b985-111ca4037344 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:32.354388 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-72bcff10-7fc9-4102-b985-111ca4037344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:32.362443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1273/2592] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:32.366658 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ac482-e687-4ef3-822d-663b42f044c8 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:32.372093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:32.378867 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ac482-e687-4ef3-822d-663b42f044c8 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:32.384938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf4c50-292c-4d97-a974-9f891f855007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:32.385586 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf4c50-292c-4d97-a974-9f891f855007 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:32.385866 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf4c50-292c-4d97-a974-9f891f855007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:32.386207 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf4c50-292c-4d97-a974-9f891f855007 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:32.386932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf4c50-292c-4d97-a974-9f891f855007 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:49:32.387240 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf4c50-292c-4d97-a974-9f891f855007 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:32.388843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf4c50-292c-4d97-a974-9f891f855007 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:32.400205 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1320/2593] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:49:32.404385 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcff10-7fc9-4102-b985-111ca4037344 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:32.413629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcff10-7fc9-4102-b985-111ca4037344 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:32.418387 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8504412-bd7e-484c-92ed-db569ea9c166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:32.419026 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8504412-bd7e-484c-92ed-db569ea9c166 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:32.419309 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8504412-bd7e-484c-92ed-db569ea9c166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:32.419587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8504412-bd7e-484c-92ed-db569ea9c166 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:32.420275 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8504412-bd7e-484c-92ed-db569ea9c166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:32.422194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8504412-bd7e-484c-92ed-db569ea9c166 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:32.447339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1274/2594] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:32 2022] POST /identity/v3/projects => generated 375 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:32.451489 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf4c50-292c-4d97-a974-9f891f855007 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:32.461750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf4c50-292c-4d97-a974-9f891f855007 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:32.522709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c400f34-b024-49e0-b49e-2b4ef8b808ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:32.523405 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c400f34-b024-49e0-b49e-2b4ef8b808ce admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:32.524256 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c400f34-b024-49e0-b49e-2b4ef8b808ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:32.524256 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c400f34-b024-49e0-b49e-2b4ef8b808ce admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:32.524821 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c400f34-b024-49e0-b49e-2b4ef8b808ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:32.528310 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c400f34-b024-49e0-b49e-2b4ef8b808ce admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:32.552893 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1321/2595] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:32 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:32.557419 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8504412-bd7e-484c-92ed-db569ea9c166 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:32.568476 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8504412-bd7e-484c-92ed-db569ea9c166 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:32.574083 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26bb70-a055-409e-8a42-b76e38a7e0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:32.574887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26bb70-a055-409e-8a42-b76e38a7e0ea admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:32.575243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26bb70-a055-409e-8a42-b76e38a7e0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:32.575594 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26bb70-a055-409e-8a42-b76e38a7e0ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:32.576332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26bb70-a055-409e-8a42-b76e38a7e0ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:32.577937 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26bb70-a055-409e-8a42-b76e38a7e0ea admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:32.589354 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1275/2596] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:32.593441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c400f34-b024-49e0-b49e-2b4ef8b808ce admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:32.605424 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c400f34-b024-49e0-b49e-2b4ef8b808ce admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:32.611629 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc61f9-04bc-4aca-a5a3-ef7ec2826aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:32.614113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc61f9-04bc-4aca-a5a3-ef7ec2826aee admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:32.614512 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc61f9-04bc-4aca-a5a3-ef7ec2826aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:32.614892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc61f9-04bc-4aca-a5a3-ef7ec2826aee admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:32.615805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc61f9-04bc-4aca-a5a3-ef7ec2826aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:32.618258 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc61f9-04bc-4aca-a5a3-ef7ec2826aee admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:32.633927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1322/2597] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:32.635259 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26bb70-a055-409e-8a42-b76e38a7e0ea admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:32.646301 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26bb70-a055-409e-8a42-b76e38a7e0ea admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:32.651409 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734328f-f07a-4658-948c-fad79dc3e810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:32.652569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734328f-f07a-4658-948c-fad79dc3e810 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:32.652856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734328f-f07a-4658-948c-fad79dc3e810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:32.653129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734328f-f07a-4658-948c-fad79dc3e810 admin admin] PATH_INFO: `/v3/projects/b6e08a151f2c4a0db9366ea417c722c1/users/0b82d36d9dbd49dfb87f4aa5537f48bf/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:32.670034 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6734328f-f07a-4658-948c-fad79dc3e810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6e08a151f2c4a0db9366ea417c722c1, user_id=0b82d36d9dbd49dfb87f4aa5537f48bf, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:32.672803 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6734328f-f07a-4658-948c-fad79dc3e810 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:32.679963 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc61f9-04bc-4aca-a5a3-ef7ec2826aee admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:32.682546 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3ddc61f9-04bc-4aca-a5a3-ef7ec2826aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:32.687888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1276/2598] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:32 2022] PUT /identity/v3/projects/b6e08a151f2c4a0db9366ea417c722c1/users/0b82d36d9dbd49dfb87f4aa5537f48bf/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:32.697768 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc61f9-04bc-4aca-a5a3-ef7ec2826aee admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:32.698531 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a6209-8929-4281-aa8c-a96dbb224506 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:32.698855 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a6209-8929-4281-aa8c-a96dbb224506 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:32.699139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a6209-8929-4281-aa8c-a96dbb224506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:32.726385 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9c9a6209-8929-4281-aa8c-a96dbb224506 None None] MFA Rules not processed for user `0b82d36d9dbd49dfb87f4aa5537f48bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:32.727557 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9c9a6209-8929-4281-aa8c-a96dbb224506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:32.756362 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9079256-ce2b-4283-bce9-3cce36e26dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:32.757410 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9079256-ce2b-4283-bce9-3cce36e26dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:32.757779 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9079256-ce2b-4283-bce9-3cce36e26dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:32.758162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9079256-ce2b-4283-bce9-3cce36e26dc9 admin admin] PATH_INFO: `/v3/users/3e60309c4d374f42ad8e83831452591b` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:32.759946 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9079256-ce2b-4283-bce9-3cce36e26dc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e60309c4d374f42ad8e83831452591b)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:32.761870 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9079256-ce2b-4283-bce9-3cce36e26dc9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:32.793949 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1277/2599] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:49:32.798596 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a6209-8929-4281-aa8c-a96dbb224506 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:32.808383 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b9079256-ce2b-4283-bce9-3cce36e26dc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e60309c4d374f42ad8e83831452591b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:32.809049 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a6209-8929-4281-aa8c-a96dbb224506 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:32.809183 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b9079256-ce2b-4283-bce9-3cce36e26dc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e60309c4d374f42ad8e83831452591b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:32.809848 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b9079256-ce2b-4283-bce9-3cce36e26dc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e60309c4d374f42ad8e83831452591b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:32.813921 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da767291-9e71-4f2f-8983-030286349685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:32.814592 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da767291-9e71-4f2f-8983-030286349685 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:32.814870 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da767291-9e71-4f2f-8983-030286349685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:32.815146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da767291-9e71-4f2f-8983-030286349685 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:32.815831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da767291-9e71-4f2f-8983-030286349685 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:32.817708 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da767291-9e71-4f2f-8983-030286349685 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:32.822886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b9079256-ce2b-4283-bce9-3cce36e26dc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e60309c4d374f42ad8e83831452591b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:32.827332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:32.830864 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b9079256-ce2b-4283-bce9-3cce36e26dc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e60309c4d374f42ad8e83831452591b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:32.848771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1278/2600] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:32 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:49:32.849748 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da767291-9e71-4f2f-8983-030286349685 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:32.850387 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1323/2601] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:32 2022] DELETE /identity/v3/users/3e60309c4d374f42ad8e83831452591b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:32.851570 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-da767291-9e71-4f2f-8983-030286349685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:32.854467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedbd83-7198-49d5-afc1-219a347bb185 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:32.854764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedbd83-7198-49d5-afc1-219a347bb185 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:32.855049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedbd83-7198-49d5-afc1-219a347bb185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:32.872890 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3cedbd83-7198-49d5-afc1-219a347bb185 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:32.874106 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3cedbd83-7198-49d5-afc1-219a347bb185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:32.879020 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:32.928062 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da767291-9e71-4f2f-8983-030286349685 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:32.939904 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1324/2602] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:32.951756 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cedbd83-7198-49d5-afc1-219a347bb185 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:32.953243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3cedbd83-7198-49d5-afc1-219a347bb185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:32.990702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6274de8-ebe2-4db6-914e-c4430dc38177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:32.991284 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6274de8-ebe2-4db6-914e-c4430dc38177 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:32.991567 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6274de8-ebe2-4db6-914e-c4430dc38177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:32.991841 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6274de8-ebe2-4db6-914e-c4430dc38177 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:32.992545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6274de8-ebe2-4db6-914e-c4430dc38177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:32.994680 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6274de8-ebe2-4db6-914e-c4430dc38177 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:33.012451 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cedbd83-7198-49d5-afc1-219a347bb185 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:33.016191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920eed1e-e801-486a-a19a-8f4ac7b4fc5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:33.016834 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920eed1e-e801-486a-a19a-8f4ac7b4fc5d service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:33.017059 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1279/2603] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:32 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:49:33.017174 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920eed1e-e801-486a-a19a-8f4ac7b4fc5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:33.017410 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920eed1e-e801-486a-a19a-8f4ac7b4fc5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:33.020602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6274de8-ebe2-4db6-914e-c4430dc38177 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:33.027665 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920eed1e-e801-486a-a19a-8f4ac7b4fc5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:33.029112 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920eed1e-e801-486a-a19a-8f4ac7b4fc5d service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:33.033413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6274de8-ebe2-4db6-914e-c4430dc38177 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:33.038160 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd644e0c-9fcf-4aac-9292-2916b0d6938c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:33.038813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd644e0c-9fcf-4aac-9292-2916b0d6938c admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:33.039072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1325/2604] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:33.039233 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd644e0c-9fcf-4aac-9292-2916b0d6938c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:33.039363 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd644e0c-9fcf-4aac-9292-2916b0d6938c admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:33.040756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd644e0c-9fcf-4aac-9292-2916b0d6938c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:33.042390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd644e0c-9fcf-4aac-9292-2916b0d6938c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:33.052816 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1280/2605] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:49:33.057309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920eed1e-e801-486a-a19a-8f4ac7b4fc5d service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:33.066920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920eed1e-e801-486a-a19a-8f4ac7b4fc5d service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:33.071841 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdc079-aa77-4f63-a9cf-991ec13de4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:33.072721 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdc079-aa77-4f63-a9cf-991ec13de4f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:33.072999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdc079-aa77-4f63-a9cf-991ec13de4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:33.073299 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdc079-aa77-4f63-a9cf-991ec13de4f8 admin admin] PATH_INFO: `/v3/projects/d7d928ee6953437e97d8d265e32498ee/users/a812309492f24b0fb9ea196515c5d8c8/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:33.090462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbdc079-aa77-4f63-a9cf-991ec13de4f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7d928ee6953437e97d8d265e32498ee, user_id=a812309492f24b0fb9ea196515c5d8c8, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:33.093291 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbdc079-aa77-4f63-a9cf-991ec13de4f8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:33.107344 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1326/2606] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:33 2022] PUT /identity/v3/projects/d7d928ee6953437e97d8d265e32498ee/users/a812309492f24b0fb9ea196515c5d8c8/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:33.111420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd644e0c-9fcf-4aac-9292-2916b0d6938c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:33.122077 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd644e0c-9fcf-4aac-9292-2916b0d6938c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:33.180984 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df4bda-a6ff-43c2-bad7-a7531ef96517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:33.181668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df4bda-a6ff-43c2-bad7-a7531ef96517 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:33.181972 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df4bda-a6ff-43c2-bad7-a7531ef96517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:33.182266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df4bda-a6ff-43c2-bad7-a7531ef96517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:33.183001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df4bda-a6ff-43c2-bad7-a7531ef96517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:33.184719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df4bda-a6ff-43c2-bad7-a7531ef96517 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:33.198804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1281/2607] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:33.199986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdc079-aa77-4f63-a9cf-991ec13de4f8 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:33.210463 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdc079-aa77-4f63-a9cf-991ec13de4f8 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:33.215544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc50069-3cbf-49a4-9d4a-fbea0316f39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:33.216424 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc50069-3cbf-49a4-9d4a-fbea0316f39d admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:33.216703 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc50069-3cbf-49a4-9d4a-fbea0316f39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:33.216996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc50069-3cbf-49a4-9d4a-fbea0316f39d admin admin] PATH_INFO: `/v3/projects/d7d928ee6953437e97d8d265e32498ee/users/a812309492f24b0fb9ea196515c5d8c8/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:33.218999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc50069-3cbf-49a4-9d4a-fbea0316f39d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7d928ee6953437e97d8d265e32498ee, user_id=a812309492f24b0fb9ea196515c5d8c8, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:33.221602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc50069-3cbf-49a4-9d4a-fbea0316f39d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:33.232653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1327/2608] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:33 2022] PUT /identity/v3/projects/d7d928ee6953437e97d8d265e32498ee/users/a812309492f24b0fb9ea196515c5d8c8/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:33.243635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e651232-e995-483e-9985-6c0141696b0a None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:33.243880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e651232-e995-483e-9985-6c0141696b0a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:33.244561 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e651232-e995-483e-9985-6c0141696b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:33.276049 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4e651232-e995-483e-9985-6c0141696b0a None None] MFA Rules not processed for user `a812309492f24b0fb9ea196515c5d8c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:33.277391 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4e651232-e995-483e-9985-6c0141696b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:33.316441 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:33.321813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc50069-3cbf-49a4-9d4a-fbea0316f39d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:33.324949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1cc50069-3cbf-49a4-9d4a-fbea0316f39d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:33.364763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1282/2609] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:33.376447 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e651232-e995-483e-9985-6c0141696b0a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:33.378228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4e651232-e995-483e-9985-6c0141696b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:33.387735 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:33.401798 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc50069-3cbf-49a4-9d4a-fbea0316f39d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:33.407257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90b926-922a-4cfd-b0cf-7386ee230b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:33.408262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90b926-922a-4cfd-b0cf-7386ee230b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:33.408735 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90b926-922a-4cfd-b0cf-7386ee230b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:33.408977 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90b926-922a-4cfd-b0cf-7386ee230b4e admin admin] PATH_INFO: `/v3/projects/3b34f9712a0041d5b81dc10d0e7a3d4e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:33.410214 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb90b926-922a-4cfd-b0cf-7386ee230b4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b34f9712a0041d5b81dc10d0e7a3d4e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:33.412130 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb90b926-922a-4cfd-b0cf-7386ee230b4e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:33.436210 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e651232-e995-483e-9985-6c0141696b0a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:33.440563 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab4dc5-c1a3-4c02-92ec-534edc7423b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:33.441296 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab4dc5-c1a3-4c02-92ec-534edc7423b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:33.441610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab4dc5-c1a3-4c02-92ec-534edc7423b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:33.441996 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab4dc5-c1a3-4c02-92ec-534edc7423b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:33.444165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f9ab4dc5-c1a3-4c02-92ec-534edc7423b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:33.461305 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-eb90b926-922a-4cfd-b0cf-7386ee230b4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b34f9712a0041d5b81dc10d0e7a3d4e', 'request_id': 'req-eb90b926-922a-4cfd-b0cf-7386ee230b4e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:33.469236 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-eb90b926-922a-4cfd-b0cf-7386ee230b4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b34f9712a0041d5b81dc10d0e7a3d4e', 'request_id': 'req-eb90b926-922a-4cfd-b0cf-7386ee230b4e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:33.476102 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-eb90b926-922a-4cfd-b0cf-7386ee230b4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b34f9712a0041d5b81dc10d0e7a3d4e', 'request_id': 'req-eb90b926-922a-4cfd-b0cf-7386ee230b4e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:33.488644 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-eb90b926-922a-4cfd-b0cf-7386ee230b4e admin admin] The token cache is being invalidate because project 3b34f9712a0041d5b81dc10d0e7a3d4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:33.488934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-eb90b926-922a-4cfd-b0cf-7386ee230b4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:33.489589 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-eb90b926-922a-4cfd-b0cf-7386ee230b4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:33.491168 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1328/2610] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:33 2022] DELETE /identity/v3/projects/3b34f9712a0041d5b81dc10d0e7a3d4e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:33.495641 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90b926-922a-4cfd-b0cf-7386ee230b4e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:33.496922 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-eb90b926-922a-4cfd-b0cf-7386ee230b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:33.506278 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab4dc5-c1a3-4c02-92ec-534edc7423b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:33.507745 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab4dc5-c1a3-4c02-92ec-534edc7423b7 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:33.516698 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1283/2611] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:49:33.562406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90b926-922a-4cfd-b0cf-7386ee230b4e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:33.567401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2434e9-7668-4328-802f-3ba3658902c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:33.568933 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2434e9-7668-4328-802f-3ba3658902c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:33.569217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2434e9-7668-4328-802f-3ba3658902c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:33.569492 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2434e9-7668-4328-802f-3ba3658902c8 admin admin] PATH_INFO: `/v3/users/386a76566f864e358a5fc0d2f6f0bda7` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:33.571403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2434e9-7668-4328-802f-3ba3658902c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=386a76566f864e358a5fc0d2f6f0bda7)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:33.573417 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2434e9-7668-4328-802f-3ba3658902c8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:33.620213 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fc2434e9-7668-4328-802f-3ba3658902c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '386a76566f864e358a5fc0d2f6f0bda7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:33.620887 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fc2434e9-7668-4328-802f-3ba3658902c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '386a76566f864e358a5fc0d2f6f0bda7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:33.621509 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fc2434e9-7668-4328-802f-3ba3658902c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '386a76566f864e358a5fc0d2f6f0bda7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:33.635515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fc2434e9-7668-4328-802f-3ba3658902c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '386a76566f864e358a5fc0d2f6f0bda7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:33.644050 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fc2434e9-7668-4328-802f-3ba3658902c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '386a76566f864e358a5fc0d2f6f0bda7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:33.658313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1329/2612] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:33 2022] DELETE /identity/v3/users/386a76566f864e358a5fc0d2f6f0bda7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:33.946660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab4dc5-c1a3-4c02-92ec-534edc7423b7 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:33.960764 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab4dc5-c1a3-4c02-92ec-534edc7423b7 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:34.018783 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b1498-8509-4140-8331-23124f5092ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:34.019646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b1498-8509-4140-8331-23124f5092ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:34.019938 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b1498-8509-4140-8331-23124f5092ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:34.020221 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b1498-8509-4140-8331-23124f5092ac admin admin] PATH_INFO: `/v3/projects/7b0f9ea2f49a4cac958fc868e48011b7` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:34.021498 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b1498-8509-4140-8331-23124f5092ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b0f9ea2f49a4cac958fc868e48011b7)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:34.023584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b1498-8509-4140-8331-23124f5092ac admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:34.053595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:34.071602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a91b1498-8509-4140-8331-23124f5092ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b0f9ea2f49a4cac958fc868e48011b7', 'request_id': 'req-a91b1498-8509-4140-8331-23124f5092ac'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:34.077018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a91b1498-8509-4140-8331-23124f5092ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b0f9ea2f49a4cac958fc868e48011b7', 'request_id': 'req-a91b1498-8509-4140-8331-23124f5092ac'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:34.084631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a91b1498-8509-4140-8331-23124f5092ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b0f9ea2f49a4cac958fc868e48011b7', 'request_id': 'req-a91b1498-8509-4140-8331-23124f5092ac'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:34.095739 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a91b1498-8509-4140-8331-23124f5092ac admin admin] The token cache is being invalidate because project 7b0f9ea2f49a4cac958fc868e48011b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:34.096027 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a91b1498-8509-4140-8331-23124f5092ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:34.096654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a91b1498-8509-4140-8331-23124f5092ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:34.098233 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1284/2613] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:33 2022] DELETE /identity/v3/projects/7b0f9ea2f49a4cac958fc868e48011b7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:34.103614 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a60e5-56f0-4c2d-87d0-b17f09077067 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:34.103932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a60e5-56f0-4c2d-87d0-b17f09077067 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:34.104212 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a60e5-56f0-4c2d-87d0-b17f09077067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:34.110476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:34.125682 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a14a60e5-56f0-4c2d-87d0-b17f09077067 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:34.127186 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a14a60e5-56f0-4c2d-87d0-b17f09077067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:34.185813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1330/2614] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:34.193578 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d8095-2d30-42ed-859c-856ec61ebab9 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:34.193896 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d8095-2d30-42ed-859c-856ec61ebab9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:34.194204 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d8095-2d30-42ed-859c-856ec61ebab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:34.212398 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-992d8095-2d30-42ed-859c-856ec61ebab9 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:34.213655 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-992d8095-2d30-42ed-859c-856ec61ebab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:34.222410 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1285/2615] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:34.226593 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a60e5-56f0-4c2d-87d0-b17f09077067 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:34.236746 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a60e5-56f0-4c2d-87d0-b17f09077067 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:34.241510 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b755a-7b44-491e-9a1c-c85c0753fe9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:34.242206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b755a-7b44-491e-9a1c-c85c0753fe9a admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:34.242511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b755a-7b44-491e-9a1c-c85c0753fe9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:34.243307 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b755a-7b44-491e-9a1c-c85c0753fe9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:34.244023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b755a-7b44-491e-9a1c-c85c0753fe9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:49:34.244331 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b755a-7b44-491e-9a1c-c85c0753fe9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:34.245953 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b755a-7b44-491e-9a1c-c85c0753fe9a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:34.257614 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1331/2616] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:49:34.262088 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d8095-2d30-42ed-859c-856ec61ebab9 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:34.271343 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d8095-2d30-42ed-859c-856ec61ebab9 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:34.276400 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe30f9-6ee4-4ff2-8c90-f8e6c493a24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:34.277023 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe30f9-6ee4-4ff2-8c90-f8e6c493a24b admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:34.277303 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe30f9-6ee4-4ff2-8c90-f8e6c493a24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:34.277574 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe30f9-6ee4-4ff2-8c90-f8e6c493a24b admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:34.278299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe30f9-6ee4-4ff2-8c90-f8e6c493a24b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:34.280156 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe30f9-6ee4-4ff2-8c90-f8e6c493a24b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:34.310047 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1286/2617] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:34 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:34.314796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b755a-7b44-491e-9a1c-c85c0753fe9a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:34.325804 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b755a-7b44-491e-9a1c-c85c0753fe9a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:34.382721 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da75a37-4a05-4e3d-ad81-bb68d84a3fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:34.383951 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da75a37-4a05-4e3d-ad81-bb68d84a3fbc admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:34.384238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da75a37-4a05-4e3d-ad81-bb68d84a3fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:34.384517 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da75a37-4a05-4e3d-ad81-bb68d84a3fbc admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:34.385201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da75a37-4a05-4e3d-ad81-bb68d84a3fbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:34.387169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da75a37-4a05-4e3d-ad81-bb68d84a3fbc admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:34.406339 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1332/2618] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:34 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:34.410348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe30f9-6ee4-4ff2-8c90-f8e6c493a24b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:34.420849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe30f9-6ee4-4ff2-8c90-f8e6c493a24b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:34.425631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f69982-c6fb-4e2b-a4a2-87e43487a263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:34.426311 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f69982-c6fb-4e2b-a4a2-87e43487a263 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:34.426593 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f69982-c6fb-4e2b-a4a2-87e43487a263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:34.426865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f69982-c6fb-4e2b-a4a2-87e43487a263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:34.427579 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f69982-c6fb-4e2b-a4a2-87e43487a263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:34.429168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f69982-c6fb-4e2b-a4a2-87e43487a263 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:34.439508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1287/2619] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:49:34.443211 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da75a37-4a05-4e3d-ad81-bb68d84a3fbc admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:34.453854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da75a37-4a05-4e3d-ad81-bb68d84a3fbc admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:34.458542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153de4e-af5c-40a7-9960-360feba6f5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:34.459195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153de4e-af5c-40a7-9960-360feba6f5ff admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:34.459484 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153de4e-af5c-40a7-9960-360feba6f5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:34.459765 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153de4e-af5c-40a7-9960-360feba6f5ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:34.460486 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153de4e-af5c-40a7-9960-360feba6f5ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:34.462735 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153de4e-af5c-40a7-9960-360feba6f5ff admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:34.472600 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1333/2620] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:34.476525 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f69982-c6fb-4e2b-a4a2-87e43487a263 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:34.486396 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f69982-c6fb-4e2b-a4a2-87e43487a263 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:34.491034 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b9a69-c39e-4104-8422-620475c6377e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:34.491902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b9a69-c39e-4104-8422-620475c6377e admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:34.492183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b9a69-c39e-4104-8422-620475c6377e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:34.492459 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b9a69-c39e-4104-8422-620475c6377e admin admin] PATH_INFO: `/v3/projects/1787bc5940cd4227904fdacfb23db726/users/26adad73c6e543cdb2fa3569354bc1cc/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:34.507229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b9a69-c39e-4104-8422-620475c6377e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1787bc5940cd4227904fdacfb23db726, user_id=26adad73c6e543cdb2fa3569354bc1cc, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:34.510608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b9a69-c39e-4104-8422-620475c6377e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:34.523813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1288/2621] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:34 2022] PUT /identity/v3/projects/1787bc5940cd4227904fdacfb23db726/users/26adad73c6e543cdb2fa3569354bc1cc/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:34.533797 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c2163-0dcd-4a09-8bc3-11caaf6ea397 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:34.534125 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c2163-0dcd-4a09-8bc3-11caaf6ea397 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:34.534406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c2163-0dcd-4a09-8bc3-11caaf6ea397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:34.560450 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-6e7c2163-0dcd-4a09-8bc3-11caaf6ea397 None None] MFA Rules not processed for user `26adad73c6e543cdb2fa3569354bc1cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:34.561646 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6e7c2163-0dcd-4a09-8bc3-11caaf6ea397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:34.619573 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:34.625911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1334/2622] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:34.630480 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b9a69-c39e-4104-8422-620475c6377e admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:34.692179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b9a69-c39e-4104-8422-620475c6377e admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:34.697489 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75d36c-4465-449d-84a2-5cd7856033be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:34.698194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75d36c-4465-449d-84a2-5cd7856033be admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:34.698477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75d36c-4465-449d-84a2-5cd7856033be admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:34.698778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75d36c-4465-449d-84a2-5cd7856033be admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:34.699475 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75d36c-4465-449d-84a2-5cd7856033be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:34.701373 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75d36c-4465-449d-84a2-5cd7856033be admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:34.725630 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1289/2623] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:34 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:34.729569 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c2163-0dcd-4a09-8bc3-11caaf6ea397 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:34.740557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c2163-0dcd-4a09-8bc3-11caaf6ea397 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:34.797863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0718c8-e1d8-4beb-ac5c-bf7c2dae8739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:34.798496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0718c8-e1d8-4beb-ac5c-bf7c2dae8739 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:34.798779 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0718c8-e1d8-4beb-ac5c-bf7c2dae8739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:34.799055 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0718c8-e1d8-4beb-ac5c-bf7c2dae8739 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:34.799741 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0718c8-e1d8-4beb-ac5c-bf7c2dae8739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:34.801674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0718c8-e1d8-4beb-ac5c-bf7c2dae8739 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:34.820537 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1335/2624] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:34 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:49:34.825160 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75d36c-4465-449d-84a2-5cd7856033be admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:34.835541 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75d36c-4465-449d-84a2-5cd7856033be admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:34.846001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226686fe-1962-4a59-a05c-816a288ad6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:34.846746 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226686fe-1962-4a59-a05c-816a288ad6c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:34.847039 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226686fe-1962-4a59-a05c-816a288ad6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:34.847351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226686fe-1962-4a59-a05c-816a288ad6c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:34.848155 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226686fe-1962-4a59-a05c-816a288ad6c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:34.849702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226686fe-1962-4a59-a05c-816a288ad6c4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:34.862711 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1290/2625] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:34.866945 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0718c8-e1d8-4beb-ac5c-bf7c2dae8739 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:34.877699 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0718c8-e1d8-4beb-ac5c-bf7c2dae8739 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:34.882403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed569c2-e0eb-44e2-a75e-9b07acf9d7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:34.883950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed569c2-e0eb-44e2-a75e-9b07acf9d7f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:34.884235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed569c2-e0eb-44e2-a75e-9b07acf9d7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:34.884510 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed569c2-e0eb-44e2-a75e-9b07acf9d7f8 admin admin] PATH_INFO: `/v3/projects/55678cef0ac649f4b4953fcb33ed7fe6/users/6621ba6c55644d88a299a95eda679107/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:35.022880 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed569c2-e0eb-44e2-a75e-9b07acf9d7f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55678cef0ac649f4b4953fcb33ed7fe6, user_id=6621ba6c55644d88a299a95eda679107, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:35.026467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed569c2-e0eb-44e2-a75e-9b07acf9d7f8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:35.048640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226686fe-1962-4a59-a05c-816a288ad6c4 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:35.049111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1336/2626] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:34 2022] PUT /identity/v3/projects/55678cef0ac649f4b4953fcb33ed7fe6/users/6621ba6c55644d88a299a95eda679107/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:35.060181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226686fe-1962-4a59-a05c-816a288ad6c4 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:35.082931 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:35.122223 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9769319-b7a4-4ee7-8ab3-42467ed8c284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:35.122895 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9769319-b7a4-4ee7-8ab3-42467ed8c284 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:35.123182 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9769319-b7a4-4ee7-8ab3-42467ed8c284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:35.123460 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9769319-b7a4-4ee7-8ab3-42467ed8c284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:35.124179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9769319-b7a4-4ee7-8ab3-42467ed8c284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:35.125785 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9769319-b7a4-4ee7-8ab3-42467ed8c284 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:35.136223 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1291/2627] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:35.140603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed569c2-e0eb-44e2-a75e-9b07acf9d7f8 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:35.151358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed569c2-e0eb-44e2-a75e-9b07acf9d7f8 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:35.155863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46d003-712c-40df-b2b3-ea5598483c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:35.156743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46d003-712c-40df-b2b3-ea5598483c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:35.157016 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46d003-712c-40df-b2b3-ea5598483c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:35.157284 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46d003-712c-40df-b2b3-ea5598483c70 admin admin] PATH_INFO: `/v3/projects/55678cef0ac649f4b4953fcb33ed7fe6/users/6621ba6c55644d88a299a95eda679107/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:35.159337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46d003-712c-40df-b2b3-ea5598483c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55678cef0ac649f4b4953fcb33ed7fe6, user_id=6621ba6c55644d88a299a95eda679107, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:35.161913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46d003-712c-40df-b2b3-ea5598483c70 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:35.172201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1337/2628] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:35 2022] PUT /identity/v3/projects/55678cef0ac649f4b4953fcb33ed7fe6/users/6621ba6c55644d88a299a95eda679107/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:35.183364 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972b1a9-1b13-4801-a4ae-1f47eeb3fc44 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:35.183654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972b1a9-1b13-4801-a4ae-1f47eeb3fc44 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:35.183936 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972b1a9-1b13-4801-a4ae-1f47eeb3fc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:35.209558 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-c972b1a9-1b13-4801-a4ae-1f47eeb3fc44 None None] MFA Rules not processed for user `6621ba6c55644d88a299a95eda679107`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:35.210799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c972b1a9-1b13-4801-a4ae-1f47eeb3fc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:35.277886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1292/2629] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:35.289288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46d003-712c-40df-b2b3-ea5598483c70 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:35.291287 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9f46d003-712c-40df-b2b3-ea5598483c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:35.347249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46d003-712c-40df-b2b3-ea5598483c70 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:35.350757 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf3e42-bb9e-4e97-be11-05c4ebfb3ec8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:35.351387 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf3e42-bb9e-4e97-be11-05c4ebfb3ec8 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:35.351667 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf3e42-bb9e-4e97-be11-05c4ebfb3ec8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:35.351940 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf3e42-bb9e-4e97-be11-05c4ebfb3ec8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:35.360665 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf3e42-bb9e-4e97-be11-05c4ebfb3ec8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:35.362212 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf3e42-bb9e-4e97-be11-05c4ebfb3ec8 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:35.372084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1338/2630] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:35.440969 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972b1a9-1b13-4801-a4ae-1f47eeb3fc44 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:35.442389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c972b1a9-1b13-4801-a4ae-1f47eeb3fc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:35.454720 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972b1a9-1b13-4801-a4ae-1f47eeb3fc44 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:35.458377 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297597d-0b5d-4e76-833c-8ae66afe9d1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:35.459007 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297597d-0b5d-4e76-833c-8ae66afe9d1c service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:35.459291 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297597d-0b5d-4e76-833c-8ae66afe9d1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:35.459567 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297597d-0b5d-4e76-833c-8ae66afe9d1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:35.514878 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297597d-0b5d-4e76-833c-8ae66afe9d1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:35.516300 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297597d-0b5d-4e76-833c-8ae66afe9d1c service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:35.525016 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1293/2631] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:35.777863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf3e42-bb9e-4e97-be11-05c4ebfb3ec8 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:35.829557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:35.841147 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf3e42-bb9e-4e97-be11-05c4ebfb3ec8 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:35.845706 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29bdbda-c250-400a-82ca-6939af222513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:35.846598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bdbda-c250-400a-82ca-6939af222513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:35.846877 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bdbda-c250-400a-82ca-6939af222513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:35.847147 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bdbda-c250-400a-82ca-6939af222513 admin admin] PATH_INFO: `/v3/users/26adad73c6e543cdb2fa3569354bc1cc` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:35.848322 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29bdbda-c250-400a-82ca-6939af222513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26adad73c6e543cdb2fa3569354bc1cc)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:35.850263 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29bdbda-c250-400a-82ca-6939af222513 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:35.897183 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a29bdbda-c250-400a-82ca-6939af222513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26adad73c6e543cdb2fa3569354bc1cc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:35.897777 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a29bdbda-c250-400a-82ca-6939af222513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26adad73c6e543cdb2fa3569354bc1cc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:35.898409 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a29bdbda-c250-400a-82ca-6939af222513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26adad73c6e543cdb2fa3569354bc1cc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:35.910474 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a29bdbda-c250-400a-82ca-6939af222513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26adad73c6e543cdb2fa3569354bc1cc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:35.917090 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a29bdbda-c250-400a-82ca-6939af222513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26adad73c6e543cdb2fa3569354bc1cc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:35.929086 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1339/2632] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:35 2022] DELETE /identity/v3/users/26adad73c6e543cdb2fa3569354bc1cc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:35.933581 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add04966-d763-458f-bae4-5bc55fb90ab0 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:35.933886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add04966-d763-458f-bae4-5bc55fb90ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:35.934192 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add04966-d763-458f-bae4-5bc55fb90ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:35.951370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-add04966-d763-458f-bae4-5bc55fb90ab0 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:35.952548 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-add04966-d763-458f-bae4-5bc55fb90ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:35.990837 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:36.011532 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1294/2633] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:36.021140 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29bdbda-c250-400a-82ca-6939af222513 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:36.022563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a29bdbda-c250-400a-82ca-6939af222513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:36.078630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29bdbda-c250-400a-82ca-6939af222513 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:36.082285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77f65b-4129-4f33-85c1-c239eb8bd0e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:36.082928 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77f65b-4129-4f33-85c1-c239eb8bd0e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:36.083210 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77f65b-4129-4f33-85c1-c239eb8bd0e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:36.083483 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77f65b-4129-4f33-85c1-c239eb8bd0e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:36.096694 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77f65b-4129-4f33-85c1-c239eb8bd0e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:36.098333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77f65b-4129-4f33-85c1-c239eb8bd0e6 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:36.110270 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1340/2634] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:36.378134 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add04966-d763-458f-bae4-5bc55fb90ab0 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:36.379588 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-add04966-d763-458f-bae4-5bc55fb90ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:36.391832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add04966-d763-458f-bae4-5bc55fb90ab0 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:36.397155 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:36.398058 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:36.398342 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:36.398611 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6 admin admin] PATH_INFO: `/v3/projects/1787bc5940cd4227904fdacfb23db726` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:36.400483 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1787bc5940cd4227904fdacfb23db726)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:36.402428 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:36.453154 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1787bc5940cd4227904fdacfb23db726', 'request_id': 'req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:36.458998 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1787bc5940cd4227904fdacfb23db726', 'request_id': 'req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:36.469019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1787bc5940cd4227904fdacfb23db726', 'request_id': 'req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:36.480122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6 admin admin] The token cache is being invalidate because project 1787bc5940cd4227904fdacfb23db726 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:36.480412 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:36.481438 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:36.483009 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1295/2635] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:36 2022] DELETE /identity/v3/projects/1787bc5940cd4227904fdacfb23db726 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:36.486854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77f65b-4129-4f33-85c1-c239eb8bd0e6 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:36.496409 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77f65b-4129-4f33-85c1-c239eb8bd0e6 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:36.551226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77efc2-9270-4de1-ad2f-b76b2382c272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:36.552098 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77efc2-9270-4de1-ad2f-b76b2382c272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:36.552386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77efc2-9270-4de1-ad2f-b76b2382c272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:36.552664 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77efc2-9270-4de1-ad2f-b76b2382c272 admin admin] PATH_INFO: `/v3/users/6621ba6c55644d88a299a95eda679107` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:36.553841 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77efc2-9270-4de1-ad2f-b76b2382c272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6621ba6c55644d88a299a95eda679107)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:36.555842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77efc2-9270-4de1-ad2f-b76b2382c272 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:36.571222 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:36.601553 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5e77efc2-9270-4de1-ad2f-b76b2382c272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6621ba6c55644d88a299a95eda679107'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:36.602217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5e77efc2-9270-4de1-ad2f-b76b2382c272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6621ba6c55644d88a299a95eda679107'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:36.602827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5e77efc2-9270-4de1-ad2f-b76b2382c272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6621ba6c55644d88a299a95eda679107'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:36.614555 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5e77efc2-9270-4de1-ad2f-b76b2382c272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6621ba6c55644d88a299a95eda679107'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:36.621195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5e77efc2-9270-4de1-ad2f-b76b2382c272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6621ba6c55644d88a299a95eda679107'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:36.633542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1341/2636] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:36 2022] DELETE /identity/v3/users/6621ba6c55644d88a299a95eda679107 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:36.913161 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:36.914792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:36.929032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7ee9a-47fc-4998-8267-d74cf1b4fea6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:36.985752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bfaef-d42c-4bc0-8da4-54930c746fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:36.986583 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bfaef-d42c-4bc0-8da4-54930c746fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:36.986864 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bfaef-d42c-4bc0-8da4-54930c746fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:36.987134 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bfaef-d42c-4bc0-8da4-54930c746fb7 admin admin] PATH_INFO: `/v3/projects/55678cef0ac649f4b4953fcb33ed7fe6` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:36.988298 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54bfaef-d42c-4bc0-8da4-54930c746fb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55678cef0ac649f4b4953fcb33ed7fe6)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:36.990222 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54bfaef-d42c-4bc0-8da4-54930c746fb7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:37.035760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a54bfaef-d42c-4bc0-8da4-54930c746fb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55678cef0ac649f4b4953fcb33ed7fe6', 'request_id': 'req-a54bfaef-d42c-4bc0-8da4-54930c746fb7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:37.041285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a54bfaef-d42c-4bc0-8da4-54930c746fb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55678cef0ac649f4b4953fcb33ed7fe6', 'request_id': 'req-a54bfaef-d42c-4bc0-8da4-54930c746fb7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:37.047792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a54bfaef-d42c-4bc0-8da4-54930c746fb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55678cef0ac649f4b4953fcb33ed7fe6', 'request_id': 'req-a54bfaef-d42c-4bc0-8da4-54930c746fb7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:37.054658 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:37.060217 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a54bfaef-d42c-4bc0-8da4-54930c746fb7 admin admin] The token cache is being invalidate because project 55678cef0ac649f4b4953fcb33ed7fe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:37.060669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a54bfaef-d42c-4bc0-8da4-54930c746fb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:37.061399 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a54bfaef-d42c-4bc0-8da4-54930c746fb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:37.063280 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1296/2637] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:36 2022] DELETE /identity/v3/projects/55678cef0ac649f4b4953fcb33ed7fe6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:37.068136 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112e29d-0645-406d-a6f0-05f4c5d2f2a6 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:37.068428 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112e29d-0645-406d-a6f0-05f4c5d2f2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:37.068705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112e29d-0645-406d-a6f0-05f4c5d2f2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:37.086769 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4112e29d-0645-406d-a6f0-05f4c5d2f2a6 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:37.087963 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4112e29d-0645-406d-a6f0-05f4c5d2f2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:37.149310 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1342/2638] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:37.157020 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fd3e2-8e87-4bb5-86b3-230509ab5e58 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:37.157328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fd3e2-8e87-4bb5-86b3-230509ab5e58 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:37.157602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fd3e2-8e87-4bb5-86b3-230509ab5e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:37.176456 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-6a3fd3e2-8e87-4bb5-86b3-230509ab5e58 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:37.177643 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6a3fd3e2-8e87-4bb5-86b3-230509ab5e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:37.187029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1297/2639] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:37.191193 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112e29d-0645-406d-a6f0-05f4c5d2f2a6 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:37.200929 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112e29d-0645-406d-a6f0-05f4c5d2f2a6 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:37.205531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a408eaf-070d-4758-8d49-5e393adf25c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:37.206215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a408eaf-070d-4758-8d49-5e393adf25c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:37.206502 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a408eaf-070d-4758-8d49-5e393adf25c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:37.206788 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a408eaf-070d-4758-8d49-5e393adf25c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:37.207535 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a408eaf-070d-4758-8d49-5e393adf25c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:49:37.207845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a408eaf-070d-4758-8d49-5e393adf25c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:37.209446 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a408eaf-070d-4758-8d49-5e393adf25c5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:37.220739 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1343/2640] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:49:37.224960 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3fd3e2-8e87-4bb5-86b3-230509ab5e58 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:37.234833 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3fd3e2-8e87-4bb5-86b3-230509ab5e58 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:37.239492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af65af-e66b-47f1-b051-a039d31528e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:37.240068 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af65af-e66b-47f1-b051-a039d31528e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:37.240348 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af65af-e66b-47f1-b051-a039d31528e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:37.240620 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af65af-e66b-47f1-b051-a039d31528e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:37.241305 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af65af-e66b-47f1-b051-a039d31528e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:37.243952 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af65af-e66b-47f1-b051-a039d31528e0 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:37.268468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1298/2641] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:37 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:49:37.272664 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a408eaf-070d-4758-8d49-5e393adf25c5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:37.282583 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a408eaf-070d-4758-8d49-5e393adf25c5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:37.339938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f9f3c-0185-44ce-8159-8cd7ab101343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:37.340599 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f9f3c-0185-44ce-8159-8cd7ab101343 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:37.340884 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f9f3c-0185-44ce-8159-8cd7ab101343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:37.341161 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f9f3c-0185-44ce-8159-8cd7ab101343 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:37.341839 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f9f3c-0185-44ce-8159-8cd7ab101343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:37.343820 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f9f3c-0185-44ce-8159-8cd7ab101343 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:37.363435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1344/2642] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:37 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:37.368171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af65af-e66b-47f1-b051-a039d31528e0 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:37.378923 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af65af-e66b-47f1-b051-a039d31528e0 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:37.383662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf9534-def6-44dd-95d8-dfae6235cc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:37.384313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf9534-def6-44dd-95d8-dfae6235cc5b admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:37.384602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf9534-def6-44dd-95d8-dfae6235cc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:37.384879 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf9534-def6-44dd-95d8-dfae6235cc5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:37.385604 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf9534-def6-44dd-95d8-dfae6235cc5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:37.387242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf9534-def6-44dd-95d8-dfae6235cc5b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:37.397560 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1299/2643] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:37.401284 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f9f3c-0185-44ce-8159-8cd7ab101343 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:37.410977 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f9f3c-0185-44ce-8159-8cd7ab101343 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:37.416025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb49bb1-6b3f-4d6e-a9e2-542637bcef33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:37.416657 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb49bb1-6b3f-4d6e-a9e2-542637bcef33 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:37.416935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb49bb1-6b3f-4d6e-a9e2-542637bcef33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:37.417203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb49bb1-6b3f-4d6e-a9e2-542637bcef33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:37.417907 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb49bb1-6b3f-4d6e-a9e2-542637bcef33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:37.419495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb49bb1-6b3f-4d6e-a9e2-542637bcef33 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:37.424495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:37.430941 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1345/2644] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:37.434944 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf9534-def6-44dd-95d8-dfae6235cc5b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:37.444804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf9534-def6-44dd-95d8-dfae6235cc5b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:37.449488 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc93977-9b58-49c1-b20d-64e03c6802da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:37.450454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc93977-9b58-49c1-b20d-64e03c6802da admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:37.450737 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc93977-9b58-49c1-b20d-64e03c6802da admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:37.451019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc93977-9b58-49c1-b20d-64e03c6802da admin admin] PATH_INFO: `/v3/projects/2338273e2d0742c5a31cd9c1bcbfd9fa/users/2587478bea9c41b6a7faf248d9e0eed8/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:37.465770 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc93977-9b58-49c1-b20d-64e03c6802da admin admin] RBAC: Authorizing `identity:create_grant(project_id=2338273e2d0742c5a31cd9c1bcbfd9fa, user_id=2587478bea9c41b6a7faf248d9e0eed8, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:37.468524 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc93977-9b58-49c1-b20d-64e03c6802da admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:37.482845 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1300/2645] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:37 2022] PUT /identity/v3/projects/2338273e2d0742c5a31cd9c1bcbfd9fa/users/2587478bea9c41b6a7faf248d9e0eed8/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:37.494139 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea7250-2b6b-408e-8d3d-b5336cab46f7 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:37.494627 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea7250-2b6b-408e-8d3d-b5336cab46f7 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:37.494752 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea7250-2b6b-408e-8d3d-b5336cab46f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:37.521533 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-abea7250-2b6b-408e-8d3d-b5336cab46f7 None None] MFA Rules not processed for user `2587478bea9c41b6a7faf248d9e0eed8`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:37.522776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-abea7250-2b6b-408e-8d3d-b5336cab46f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:37.584170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1346/2646] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:37.595869 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc93977-9b58-49c1-b20d-64e03c6802da admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:37.597558 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1bc93977-9b58-49c1-b20d-64e03c6802da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:37.654648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc93977-9b58-49c1-b20d-64e03c6802da admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:37.658323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa442b-e82f-439f-aa3e-dac4f57035a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:37.659210 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa442b-e82f-439f-aa3e-dac4f57035a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:37.659525 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa442b-e82f-439f-aa3e-dac4f57035a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:37.659901 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa442b-e82f-439f-aa3e-dac4f57035a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:37.670122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa442b-e82f-439f-aa3e-dac4f57035a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:37.671556 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa442b-e82f-439f-aa3e-dac4f57035a5 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:37.680253 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1301/2647] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:49:37.698746 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea7250-2b6b-408e-8d3d-b5336cab46f7 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:37.758799 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea7250-2b6b-408e-8d3d-b5336cab46f7 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:37.763549 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd313317-a35b-4ea3-8499-3caec676179e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:37.764411 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd313317-a35b-4ea3-8499-3caec676179e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:37.764692 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd313317-a35b-4ea3-8499-3caec676179e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:37.764966 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd313317-a35b-4ea3-8499-3caec676179e admin admin] PATH_INFO: `/v3/users/2587478bea9c41b6a7faf248d9e0eed8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:37.766180 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd313317-a35b-4ea3-8499-3caec676179e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2587478bea9c41b6a7faf248d9e0eed8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:37.768099 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd313317-a35b-4ea3-8499-3caec676179e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:37.813029 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-bd313317-a35b-4ea3-8499-3caec676179e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2587478bea9c41b6a7faf248d9e0eed8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:37.813691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-bd313317-a35b-4ea3-8499-3caec676179e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2587478bea9c41b6a7faf248d9e0eed8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:37.814321 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-bd313317-a35b-4ea3-8499-3caec676179e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2587478bea9c41b6a7faf248d9e0eed8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:37.826802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-bd313317-a35b-4ea3-8499-3caec676179e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2587478bea9c41b6a7faf248d9e0eed8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:37.834475 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-bd313317-a35b-4ea3-8499-3caec676179e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2587478bea9c41b6a7faf248d9e0eed8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:37.856924 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1347/2648] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:37 2022] DELETE /identity/v3/users/2587478bea9c41b6a7faf248d9e0eed8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:37.861386 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1805c-b757-49f6-8173-1133338c2916 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:37.861730 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1805c-b757-49f6-8173-1133338c2916 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:37.862043 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1805c-b757-49f6-8173-1133338c2916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:37.879176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4ac1805c-b757-49f6-8173-1133338c2916 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:37.880372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4ac1805c-b757-49f6-8173-1133338c2916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:37.938699 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1302/2649] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:37.949090 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd313317-a35b-4ea3-8499-3caec676179e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:37.950556 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-bd313317-a35b-4ea3-8499-3caec676179e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:37.962121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:38.007775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd313317-a35b-4ea3-8499-3caec676179e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:38.012095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5eb02-fbe8-4013-81c9-638d4b90ed09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:38.012676 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5eb02-fbe8-4013-81c9-638d4b90ed09 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:38.012961 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5eb02-fbe8-4013-81c9-638d4b90ed09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:38.013236 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5eb02-fbe8-4013-81c9-638d4b90ed09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:38.022234 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5eb02-fbe8-4013-81c9-638d4b90ed09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:38.023682 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5eb02-fbe8-4013-81c9-638d4b90ed09 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:38.033493 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1348/2650] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:38.299471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1805c-b757-49f6-8173-1133338c2916 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:38.301033 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4ac1805c-b757-49f6-8173-1133338c2916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:38.314517 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1805c-b757-49f6-8173-1133338c2916 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:38.319203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e172beb-9b2b-4a89-a3d8-346c5013f14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:38.320076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172beb-9b2b-4a89-a3d8-346c5013f14f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:38.320365 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172beb-9b2b-4a89-a3d8-346c5013f14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:38.320694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172beb-9b2b-4a89-a3d8-346c5013f14f admin admin] PATH_INFO: `/v3/projects/2338273e2d0742c5a31cd9c1bcbfd9fa` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:38.321896 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e172beb-9b2b-4a89-a3d8-346c5013f14f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2338273e2d0742c5a31cd9c1bcbfd9fa)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:38.323854 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e172beb-9b2b-4a89-a3d8-346c5013f14f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:38.360239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:38.374326 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1e172beb-9b2b-4a89-a3d8-346c5013f14f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2338273e2d0742c5a31cd9c1bcbfd9fa', 'request_id': 'req-1e172beb-9b2b-4a89-a3d8-346c5013f14f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:38.380421 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1e172beb-9b2b-4a89-a3d8-346c5013f14f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2338273e2d0742c5a31cd9c1bcbfd9fa', 'request_id': 'req-1e172beb-9b2b-4a89-a3d8-346c5013f14f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:38.387195 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1e172beb-9b2b-4a89-a3d8-346c5013f14f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2338273e2d0742c5a31cd9c1bcbfd9fa', 'request_id': 'req-1e172beb-9b2b-4a89-a3d8-346c5013f14f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:38.398211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1e172beb-9b2b-4a89-a3d8-346c5013f14f admin admin] The token cache is being invalidate because project 2338273e2d0742c5a31cd9c1bcbfd9fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:38.398508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1e172beb-9b2b-4a89-a3d8-346c5013f14f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:38.399637 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1e172beb-9b2b-4a89-a3d8-346c5013f14f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:38.401187 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1303/2651] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:38 2022] DELETE /identity/v3/projects/2338273e2d0742c5a31cd9c1bcbfd9fa => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:38.406295 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd58ca2-b275-4e76-9631-23e487f4f9db None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:38.406588 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd58ca2-b275-4e76-9631-23e487f4f9db None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:38.406869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd58ca2-b275-4e76-9631-23e487f4f9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:38.424750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ddd58ca2-b275-4e76-9631-23e487f4f9db None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:38.425935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ddd58ca2-b275-4e76-9631-23e487f4f9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:38.482236 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1349/2652] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:38.547522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d32a1-c776-4fe6-99a8-2a65860b07c7 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:38.547718 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d32a1-c776-4fe6-99a8-2a65860b07c7 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:38.548033 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d32a1-c776-4fe6-99a8-2a65860b07c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:38.568677 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e99d32a1-c776-4fe6-99a8-2a65860b07c7 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:38.570079 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e99d32a1-c776-4fe6-99a8-2a65860b07c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:38.581074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1304/2653] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:38.585164 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd58ca2-b275-4e76-9631-23e487f4f9db None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:38.596019 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd58ca2-b275-4e76-9631-23e487f4f9db None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:38.603461 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ee710-0e6e-4fea-bd3d-3c577870f294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:38.604875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ee710-0e6e-4fea-bd3d-3c577870f294 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:38.605238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ee710-0e6e-4fea-bd3d-3c577870f294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:38.606189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ee710-0e6e-4fea-bd3d-3c577870f294 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:38.607009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ee710-0e6e-4fea-bd3d-3c577870f294 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:49:38.607329 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ee710-0e6e-4fea-bd3d-3c577870f294 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:38.608983 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ee710-0e6e-4fea-bd3d-3c577870f294 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:38.629555 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1350/2654] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:49:38.633525 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d32a1-c776-4fe6-99a8-2a65860b07c7 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:38.643074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d32a1-c776-4fe6-99a8-2a65860b07c7 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:38.647840 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb11bf8-8f55-4dc8-b4e3-a39c03cf5d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:38.648497 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb11bf8-8f55-4dc8-b4e3-a39c03cf5d25 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:38.648810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb11bf8-8f55-4dc8-b4e3-a39c03cf5d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:38.649088 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb11bf8-8f55-4dc8-b4e3-a39c03cf5d25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:38.649788 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb11bf8-8f55-4dc8-b4e3-a39c03cf5d25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:38.651715 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb11bf8-8f55-4dc8-b4e3-a39c03cf5d25 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:38.677926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1305/2655] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:38 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:49:38.682683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ee710-0e6e-4fea-bd3d-3c577870f294 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:38.692574 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ee710-0e6e-4fea-bd3d-3c577870f294 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:38.748840 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e170a-2749-4da3-b7fa-c47e8d708323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:38.749506 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e170a-2749-4da3-b7fa-c47e8d708323 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:38.749789 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e170a-2749-4da3-b7fa-c47e8d708323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:38.750095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e170a-2749-4da3-b7fa-c47e8d708323 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:38.750776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e170a-2749-4da3-b7fa-c47e8d708323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:38.752939 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e170a-2749-4da3-b7fa-c47e8d708323 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:38.774445 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1351/2656] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:38 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:38.778507 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb11bf8-8f55-4dc8-b4e3-a39c03cf5d25 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:38.788715 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb11bf8-8f55-4dc8-b4e3-a39c03cf5d25 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:38.793337 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023de0d-c63f-40af-ad2d-ff48453a3b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:38.794014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023de0d-c63f-40af-ad2d-ff48453a3b66 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:38.794299 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023de0d-c63f-40af-ad2d-ff48453a3b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:38.794573 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023de0d-c63f-40af-ad2d-ff48453a3b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:38.795290 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023de0d-c63f-40af-ad2d-ff48453a3b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:38.796877 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023de0d-c63f-40af-ad2d-ff48453a3b66 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:38.807116 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1306/2657] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:38.810967 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e170a-2749-4da3-b7fa-c47e8d708323 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:38.821606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e170a-2749-4da3-b7fa-c47e8d708323 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:38.826285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4b39a-d683-4d37-9181-4c5f2fc70bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:38.826952 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4b39a-d683-4d37-9181-4c5f2fc70bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:38.827238 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4b39a-d683-4d37-9181-4c5f2fc70bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:38.827517 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4b39a-d683-4d37-9181-4c5f2fc70bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:38.828239 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4b39a-d683-4d37-9181-4c5f2fc70bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:38.829892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4b39a-d683-4d37-9181-4c5f2fc70bf0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:38.840413 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1352/2658] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:38.844905 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023de0d-c63f-40af-ad2d-ff48453a3b66 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:38.855642 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023de0d-c63f-40af-ad2d-ff48453a3b66 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:38.860383 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5cc15-72ed-4235-bd4d-ceca3bdb60a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:38.861275 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5cc15-72ed-4235-bd4d-ceca3bdb60a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:38.861548 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5cc15-72ed-4235-bd4d-ceca3bdb60a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:38.861822 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5cc15-72ed-4235-bd4d-ceca3bdb60a6 admin admin] PATH_INFO: `/v3/projects/0d8f85eb9a67497e9d4307e9a7f0bd9a/users/fb912c424c2d4ae1a43b2a13aac0def2/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:38.876914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5cc15-72ed-4235-bd4d-ceca3bdb60a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d8f85eb9a67497e9d4307e9a7f0bd9a, user_id=fb912c424c2d4ae1a43b2a13aac0def2, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:38.879832 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5cc15-72ed-4235-bd4d-ceca3bdb60a6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:38.893580 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1307/2659] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:38 2022] PUT /identity/v3/projects/0d8f85eb9a67497e9d4307e9a7f0bd9a/users/fb912c424c2d4ae1a43b2a13aac0def2/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:38.904415 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25256b99-ec89-4cf8-b025-25130c997540 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:38.904710 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25256b99-ec89-4cf8-b025-25130c997540 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:38.904994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25256b99-ec89-4cf8-b025-25130c997540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:38.930462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-25256b99-ec89-4cf8-b025-25130c997540 None None] MFA Rules not processed for user `fb912c424c2d4ae1a43b2a13aac0def2`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:38.931678 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-25256b99-ec89-4cf8-b025-25130c997540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:38.936621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:39.002931 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1353/2660] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:39.007599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5cc15-72ed-4235-bd4d-ceca3bdb60a6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:39.072951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5cc15-72ed-4235-bd4d-ceca3bdb60a6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:39.078305 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250f179-872e-45dc-83cb-0ff7bdc9a243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:39.079325 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250f179-872e-45dc-83cb-0ff7bdc9a243 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:39.079614 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250f179-872e-45dc-83cb-0ff7bdc9a243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:39.079889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250f179-872e-45dc-83cb-0ff7bdc9a243 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:39.080595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a250f179-872e-45dc-83cb-0ff7bdc9a243 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:39.082545 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a250f179-872e-45dc-83cb-0ff7bdc9a243 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:39.110789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1308/2661] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:39 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:39.114611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25256b99-ec89-4cf8-b025-25130c997540 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:39.125705 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25256b99-ec89-4cf8-b025-25130c997540 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:39.187253 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351c99d-0790-423a-b194-1973a1a00e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:39.187930 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351c99d-0790-423a-b194-1973a1a00e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:39.188219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351c99d-0790-423a-b194-1973a1a00e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:39.188497 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351c99d-0790-423a-b194-1973a1a00e93 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:39.189191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351c99d-0790-423a-b194-1973a1a00e93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:39.191162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351c99d-0790-423a-b194-1973a1a00e93 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:39.203161 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250f179-872e-45dc-83cb-0ff7bdc9a243 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:39.205096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a250f179-872e-45dc-83cb-0ff7bdc9a243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:39.215465 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1354/2662] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:39 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:39.216491 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351c99d-0790-423a-b194-1973a1a00e93 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:39.226371 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351c99d-0790-423a-b194-1973a1a00e93 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:39.230996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9eebe6-136c-494b-94e8-5003fac736bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:39.231638 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eebe6-136c-494b-94e8-5003fac736bd admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:39.231920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eebe6-136c-494b-94e8-5003fac736bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:39.232441 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eebe6-136c-494b-94e8-5003fac736bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:39.232897 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9eebe6-136c-494b-94e8-5003fac736bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:39.234518 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9eebe6-136c-494b-94e8-5003fac736bd admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:39.251193 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1355/2663] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:49:39.252606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9eebe6-136c-494b-94e8-5003fac736bd admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:39.253938 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:39.265570 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9eebe6-136c-494b-94e8-5003fac736bd admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:39.271391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b234da-60cc-4b73-83bc-02508bc217c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:39.272089 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b234da-60cc-4b73-83bc-02508bc217c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:39.272233 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250f179-872e-45dc-83cb-0ff7bdc9a243 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:39.272406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b234da-60cc-4b73-83bc-02508bc217c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:39.272644 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b234da-60cc-4b73-83bc-02508bc217c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:39.273370 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b234da-60cc-4b73-83bc-02508bc217c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:39.275022 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b234da-60cc-4b73-83bc-02508bc217c0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:39.276946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd135c-dba5-44cd-adc0-a87144cc5193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:39.277594 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd135c-dba5-44cd-adc0-a87144cc5193 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:39.277939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd135c-dba5-44cd-adc0-a87144cc5193 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:39.278264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd135c-dba5-44cd-adc0-a87144cc5193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:39.280053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1fcd135c-dba5-44cd-adc0-a87144cc5193 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:39.287822 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1356/2664] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:49:39.293059 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b234da-60cc-4b73-83bc-02508bc217c0 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:39.310118 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b234da-60cc-4b73-83bc-02508bc217c0 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:39.316084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36851a0-cb3b-4b74-b954-f30288e57aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:39.317104 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36851a0-cb3b-4b74-b954-f30288e57aad admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:39.317450 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36851a0-cb3b-4b74-b954-f30288e57aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:39.317733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36851a0-cb3b-4b74-b954-f30288e57aad admin admin] PATH_INFO: `/v3/projects/50e6a4397dc84724864bc6f5f1151126/users/1a2f3c81f15e47138a70e7fcdb12c6c4/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:39.333781 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36851a0-cb3b-4b74-b954-f30288e57aad admin admin] RBAC: Authorizing `identity:create_grant(project_id=50e6a4397dc84724864bc6f5f1151126, user_id=1a2f3c81f15e47138a70e7fcdb12c6c4, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:39.336493 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36851a0-cb3b-4b74-b954-f30288e57aad admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:39.351393 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1357/2665] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:39 2022] PUT /identity/v3/projects/50e6a4397dc84724864bc6f5f1151126/users/1a2f3c81f15e47138a70e7fcdb12c6c4/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:49:39.359535 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd135c-dba5-44cd-adc0-a87144cc5193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:39.360965 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd135c-dba5-44cd-adc0-a87144cc5193 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:39.363669 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3102d05-383b-4b86-935d-b978c2dbe299 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:39.363964 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3102d05-383b-4b86-935d-b978c2dbe299 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:39.364271 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3102d05-383b-4b86-935d-b978c2dbe299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:39.372289 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1309/2666] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:49:39.393462 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-f3102d05-383b-4b86-935d-b978c2dbe299 None None] MFA Rules not processed for user `1a2f3c81f15e47138a70e7fcdb12c6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:39.394719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f3102d05-383b-4b86-935d-b978c2dbe299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:39.436723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:39.473230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1358/2667] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:49:39.478889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd135c-dba5-44cd-adc0-a87144cc5193 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:39.491644 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd135c-dba5-44cd-adc0-a87144cc5193 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:39.576117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d9bcb-dc16-4bdd-b1ab-9e54ab7beae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:39.577078 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d9bcb-dc16-4bdd-b1ab-9e54ab7beae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:39.577345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d9bcb-dc16-4bdd-b1ab-9e54ab7beae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:39.577741 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d9bcb-dc16-4bdd-b1ab-9e54ab7beae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:39.578479 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d9bcb-dc16-4bdd-b1ab-9e54ab7beae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:39.580394 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d9bcb-dc16-4bdd-b1ab-9e54ab7beae4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:39.614407 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1310/2668] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:49:39 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:39.617179 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3102d05-383b-4b86-935d-b978c2dbe299 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:39.633327 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3102d05-383b-4b86-935d-b978c2dbe299 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:39.650745 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d9bcb-dc16-4bdd-b1ab-9e54ab7beae4 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:39.655339 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0c3d9bcb-dc16-4bdd-b1ab-9e54ab7beae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:39.707881 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a3387-14e9-4ceb-840d-a9534a59f856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:39.709076 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a3387-14e9-4ceb-840d-a9534a59f856 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:39.709558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a3387-14e9-4ceb-840d-a9534a59f856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:39.710901 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a3387-14e9-4ceb-840d-a9534a59f856 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:39.723194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a3387-14e9-4ceb-840d-a9534a59f856 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:39.723194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a3387-14e9-4ceb-840d-a9534a59f856 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:39.754367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1359/2669] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:49:39 2022] POST /identity/v3/users => generated 299 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:49:39.758476 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a3387-14e9-4ceb-840d-a9534a59f856 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:39.766005 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d9bcb-dc16-4bdd-b1ab-9e54ab7beae4 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:39.771365 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99df56e-e566-4b67-80be-3d92ff9bf44c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:39.772053 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a3387-14e9-4ceb-840d-a9534a59f856 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:39.772201 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99df56e-e566-4b67-80be-3d92ff9bf44c service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:39.772387 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99df56e-e566-4b67-80be-3d92ff9bf44c service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:39.772698 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99df56e-e566-4b67-80be-3d92ff9bf44c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:39.775098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e99df56e-e566-4b67-80be-3d92ff9bf44c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:39.780331 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210a552-f3ec-4dae-9597-c0ec6b43470e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:39.781076 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210a552-f3ec-4dae-9597-c0ec6b43470e admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:39.781436 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210a552-f3ec-4dae-9597-c0ec6b43470e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:39.781805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210a552-f3ec-4dae-9597-c0ec6b43470e admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:39.782806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210a552-f3ec-4dae-9597-c0ec6b43470e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:39.784506 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210a552-f3ec-4dae-9597-c0ec6b43470e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:39.786049 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99df56e-e566-4b67-80be-3d92ff9bf44c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:39.787625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99df56e-e566-4b67-80be-3d92ff9bf44c service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:39.797994 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1311/2670] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:49:39.801077 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1360/2671] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:49:39.802173 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99df56e-e566-4b67-80be-3d92ff9bf44c service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:39.814560 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99df56e-e566-4b67-80be-3d92ff9bf44c service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:39.820191 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b5cf6-23cb-47ee-86a8-bb4378ec5668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:39.821747 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b5cf6-23cb-47ee-86a8-bb4378ec5668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:39.822776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b5cf6-23cb-47ee-86a8-bb4378ec5668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:39.823068 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b5cf6-23cb-47ee-86a8-bb4378ec5668 admin admin] PATH_INFO: `/v3/projects/e5f0bca636ae47a9b4c0edbb38a0cdf0/users/05a64f7b37694096a79b2822d1e023df/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:39.840190 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b5cf6-23cb-47ee-86a8-bb4378ec5668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5f0bca636ae47a9b4c0edbb38a0cdf0, user_id=05a64f7b37694096a79b2822d1e023df, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:39.846345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b5cf6-23cb-47ee-86a8-bb4378ec5668 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:39.856338 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:39.864779 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1312/2672] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:39 2022] PUT /identity/v3/projects/e5f0bca636ae47a9b4c0edbb38a0cdf0/users/05a64f7b37694096a79b2822d1e023df/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:39.869831 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210a552-f3ec-4dae-9597-c0ec6b43470e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:39.884137 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210a552-f3ec-4dae-9597-c0ec6b43470e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:39.952541 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f30103-391f-4119-975e-03328c476c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:39.954115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f30103-391f-4119-975e-03328c476c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:39.954115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f30103-391f-4119-975e-03328c476c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:39.954115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f30103-391f-4119-975e-03328c476c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:39.954538 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f30103-391f-4119-975e-03328c476c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:39.956139 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f30103-391f-4119-975e-03328c476c20 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:39.967796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1361/2673] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:49:39.972217 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b5cf6-23cb-47ee-86a8-bb4378ec5668 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:39.985690 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b5cf6-23cb-47ee-86a8-bb4378ec5668 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:39.995591 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adccac-051e-49c4-abb5-74d6eaee2ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:39.997280 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adccac-051e-49c4-abb5-74d6eaee2ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:39.997441 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adccac-051e-49c4-abb5-74d6eaee2ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:39.998067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adccac-051e-49c4-abb5-74d6eaee2ac2 admin admin] PATH_INFO: `/v3/projects/e5f0bca636ae47a9b4c0edbb38a0cdf0/users/05a64f7b37694096a79b2822d1e023df/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:39.998290 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f30103-391f-4119-975e-03328c476c20 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:39.999973 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9adccac-051e-49c4-abb5-74d6eaee2ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5f0bca636ae47a9b4c0edbb38a0cdf0, user_id=05a64f7b37694096a79b2822d1e023df, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:40.000101 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c3f30103-391f-4119-975e-03328c476c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:40.003155 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9adccac-051e-49c4-abb5-74d6eaee2ac2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:40.014884 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1313/2674] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:49:39 2022] PUT /identity/v3/projects/e5f0bca636ae47a9b4c0edbb38a0cdf0/users/05a64f7b37694096a79b2822d1e023df/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:40.026239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c5a13-2e49-4801-bc5b-cc72f65f9bb8 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:40.026591 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c5a13-2e49-4801-bc5b-cc72f65f9bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:40.026872 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c5a13-2e49-4801-bc5b-cc72f65f9bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:40.056010 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7c2c5a13-2e49-4801-bc5b-cc72f65f9bb8 None None] MFA Rules not processed for user `05a64f7b37694096a79b2822d1e023df`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:40.057760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7c2c5a13-2e49-4801-bc5b-cc72f65f9bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:40.069598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f30103-391f-4119-975e-03328c476c20 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:40.078982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:40.131872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935b423-fec3-4aeb-b90e-cfc94fb62b27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:40.132528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935b423-fec3-4aeb-b90e-cfc94fb62b27 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:40.133446 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935b423-fec3-4aeb-b90e-cfc94fb62b27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:40.133717 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935b423-fec3-4aeb-b90e-cfc94fb62b27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:40.135785 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d935b423-fec3-4aeb-b90e-cfc94fb62b27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:40.137747 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1314/2675] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:49:40.154879 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c5a13-2e49-4801-bc5b-cc72f65f9bb8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:40.216895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d935b423-fec3-4aeb-b90e-cfc94fb62b27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:40.218634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d935b423-fec3-4aeb-b90e-cfc94fb62b27 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:40.225190 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c5a13-2e49-4801-bc5b-cc72f65f9bb8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:40.229664 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1362/2676] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:49:40.229829 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b713bfd-1e5e-481f-9de5-22d9bc61361c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:40.230732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b713bfd-1e5e-481f-9de5-22d9bc61361c service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:40.231019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b713bfd-1e5e-481f-9de5-22d9bc61361c service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:40.231356 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b713bfd-1e5e-481f-9de5-22d9bc61361c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:40.304312 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b713bfd-1e5e-481f-9de5-22d9bc61361c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:40.306612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b713bfd-1e5e-481f-9de5-22d9bc61361c service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:40.318925 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1315/2677] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:44.091655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935b423-fec3-4aeb-b90e-cfc94fb62b27 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:44.104783 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935b423-fec3-4aeb-b90e-cfc94fb62b27 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:44.109998 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fe54f-6a7a-4600-ac2c-a3a20944bcff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:44.110889 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fe54f-6a7a-4600-ac2c-a3a20944bcff service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:44.111314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fe54f-6a7a-4600-ac2c-a3a20944bcff service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:44.111669 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fe54f-6a7a-4600-ac2c-a3a20944bcff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:44.267167 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655fe54f-6a7a-4600-ac2c-a3a20944bcff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:44.269489 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655fe54f-6a7a-4600-ac2c-a3a20944bcff service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:44.282444 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1363/2678] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:45.039277 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b713bfd-1e5e-481f-9de5-22d9bc61361c service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:45.050971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b713bfd-1e5e-481f-9de5-22d9bc61361c service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:45.056359 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62953b23-3cf3-48c7-bd9c-c3ca8ffac988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:45.057148 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62953b23-3cf3-48c7-bd9c-c3ca8ffac988 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:45.057601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62953b23-3cf3-48c7-bd9c-c3ca8ffac988 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:45.057947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62953b23-3cf3-48c7-bd9c-c3ca8ffac988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:45.071928 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62953b23-3cf3-48c7-bd9c-c3ca8ffac988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:45.073373 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62953b23-3cf3-48c7-bd9c-c3ca8ffac988 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:45.101950 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1316/2679] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:51.100390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fe54f-6a7a-4600-ac2c-a3a20944bcff service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:51.163091 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fe54f-6a7a-4600-ac2c-a3a20944bcff service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:51.167689 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf10bb8-4824-43c1-bdf9-6f6b5c64783f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:51.168581 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf10bb8-4824-43c1-bdf9-6f6b5c64783f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:51.168866 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf10bb8-4824-43c1-bdf9-6f6b5c64783f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:51.169139 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf10bb8-4824-43c1-bdf9-6f6b5c64783f admin admin] PATH_INFO: `/v3/users/fb912c424c2d4ae1a43b2a13aac0def2` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:51.170370 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf10bb8-4824-43c1-bdf9-6f6b5c64783f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb912c424c2d4ae1a43b2a13aac0def2)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:51.172281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf10bb8-4824-43c1-bdf9-6f6b5c64783f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:51.173742 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:51.217391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-0cf10bb8-4824-43c1-bdf9-6f6b5c64783f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb912c424c2d4ae1a43b2a13aac0def2'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:51.218091 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-0cf10bb8-4824-43c1-bdf9-6f6b5c64783f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb912c424c2d4ae1a43b2a13aac0def2'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:51.218699 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-0cf10bb8-4824-43c1-bdf9-6f6b5c64783f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb912c424c2d4ae1a43b2a13aac0def2'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:51.230411 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-0cf10bb8-4824-43c1-bdf9-6f6b5c64783f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb912c424c2d4ae1a43b2a13aac0def2'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:51.237150 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-0cf10bb8-4824-43c1-bdf9-6f6b5c64783f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb912c424c2d4ae1a43b2a13aac0def2'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:51.251361 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1364/2680] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:51 2022] DELETE /identity/v3/users/fb912c424c2d4ae1a43b2a13aac0def2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:51.256133 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7d273-3d02-490d-aa40-817efa81bbb4 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:51.256632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7d273-3d02-490d-aa40-817efa81bbb4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:51.256757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7d273-3d02-490d-aa40-817efa81bbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:51.274114 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9df7d273-3d02-490d-aa40-817efa81bbb4 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:49:51.275317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9df7d273-3d02-490d-aa40-817efa81bbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:51.279224 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:51.334986 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1317/2681] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:49:51.344435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf10bb8-4824-43c1-bdf9-6f6b5c64783f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:51.345841 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0cf10bb8-4824-43c1-bdf9-6f6b5c64783f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:51.403741 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf10bb8-4824-43c1-bdf9-6f6b5c64783f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:51.407313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d82bbf-08ac-4993-ac49-e5782b27cd23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:51.407949 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d82bbf-08ac-4993-ac49-e5782b27cd23 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:51.408232 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d82bbf-08ac-4993-ac49-e5782b27cd23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:51.408509 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d82bbf-08ac-4993-ac49-e5782b27cd23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:51.417343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d82bbf-08ac-4993-ac49-e5782b27cd23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:51.418796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d82bbf-08ac-4993-ac49-e5782b27cd23 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:51.428025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1365/2682] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:49:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:49:51.703115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7d273-3d02-490d-aa40-817efa81bbb4 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:51.703115 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9df7d273-3d02-490d-aa40-817efa81bbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:51.741126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7d273-3d02-490d-aa40-817efa81bbb4 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:51.746243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc472d5-577d-402b-bed3-5dcb504fe8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:51.747129 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc472d5-577d-402b-bed3-5dcb504fe8e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:51.747411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc472d5-577d-402b-bed3-5dcb504fe8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:51.747685 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc472d5-577d-402b-bed3-5dcb504fe8e2 admin admin] PATH_INFO: `/v3/projects/0d8f85eb9a67497e9d4307e9a7f0bd9a` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:51.748847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc472d5-577d-402b-bed3-5dcb504fe8e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d8f85eb9a67497e9d4307e9a7f0bd9a)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:51.750848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc472d5-577d-402b-bed3-5dcb504fe8e2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:51.796351 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3bc472d5-577d-402b-bed3-5dcb504fe8e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d8f85eb9a67497e9d4307e9a7f0bd9a', 'request_id': 'req-3bc472d5-577d-402b-bed3-5dcb504fe8e2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:51.801896 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3bc472d5-577d-402b-bed3-5dcb504fe8e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d8f85eb9a67497e9d4307e9a7f0bd9a', 'request_id': 'req-3bc472d5-577d-402b-bed3-5dcb504fe8e2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:51.810015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3bc472d5-577d-402b-bed3-5dcb504fe8e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d8f85eb9a67497e9d4307e9a7f0bd9a', 'request_id': 'req-3bc472d5-577d-402b-bed3-5dcb504fe8e2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:51.821345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3bc472d5-577d-402b-bed3-5dcb504fe8e2 admin admin] The token cache is being invalidate because project 0d8f85eb9a67497e9d4307e9a7f0bd9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:51.821631 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3bc472d5-577d-402b-bed3-5dcb504fe8e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:51.822267 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3bc472d5-577d-402b-bed3-5dcb504fe8e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:51.823792 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1318/2683] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:51 2022] DELETE /identity/v3/projects/0d8f85eb9a67497e9d4307e9a7f0bd9a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:51.827768 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d82bbf-08ac-4993-ac49-e5782b27cd23 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:51.878842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d82bbf-08ac-4993-ac49-e5782b27cd23 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:51.916533 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:51.934071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e145980-85bc-4f64-a2b6-e24146e76b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:51.934966 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e145980-85bc-4f64-a2b6-e24146e76b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:51.935249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e145980-85bc-4f64-a2b6-e24146e76b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:51.935524 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e145980-85bc-4f64-a2b6-e24146e76b70 admin admin] PATH_INFO: `/v3/users/1a2f3c81f15e47138a70e7fcdb12c6c4` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:51.936723 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e145980-85bc-4f64-a2b6-e24146e76b70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a2f3c81f15e47138a70e7fcdb12c6c4)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:51.938682 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e145980-85bc-4f64-a2b6-e24146e76b70 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:51.982630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2e145980-85bc-4f64-a2b6-e24146e76b70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a2f3c81f15e47138a70e7fcdb12c6c4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:51.983260 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2e145980-85bc-4f64-a2b6-e24146e76b70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a2f3c81f15e47138a70e7fcdb12c6c4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:51.983843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2e145980-85bc-4f64-a2b6-e24146e76b70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a2f3c81f15e47138a70e7fcdb12c6c4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:51.996127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2e145980-85bc-4f64-a2b6-e24146e76b70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a2f3c81f15e47138a70e7fcdb12c6c4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:52.003017 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2e145980-85bc-4f64-a2b6-e24146e76b70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a2f3c81f15e47138a70e7fcdb12c6c4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:52.016025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1366/2684] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:51 2022] DELETE /identity/v3/users/1a2f3c81f15e47138a70e7fcdb12c6c4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:52.370293 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc472d5-577d-402b-bed3-5dcb504fe8e2 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:52.371613 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3bc472d5-577d-402b-bed3-5dcb504fe8e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:52.419878 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc472d5-577d-402b-bed3-5dcb504fe8e2 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:52.456979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:52.476875 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26277ac-a5ad-448c-a4e1-4ec064b014fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:52.477775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26277ac-a5ad-448c-a4e1-4ec064b014fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:52.478093 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26277ac-a5ad-448c-a4e1-4ec064b014fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:52.478368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26277ac-a5ad-448c-a4e1-4ec064b014fb admin admin] PATH_INFO: `/v3/projects/50e6a4397dc84724864bc6f5f1151126` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:52.480193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26277ac-a5ad-448c-a4e1-4ec064b014fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=50e6a4397dc84724864bc6f5f1151126)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:52.482158 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26277ac-a5ad-448c-a4e1-4ec064b014fb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:52.526526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a26277ac-a5ad-448c-a4e1-4ec064b014fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50e6a4397dc84724864bc6f5f1151126', 'request_id': 'req-a26277ac-a5ad-448c-a4e1-4ec064b014fb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:52.531982 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a26277ac-a5ad-448c-a4e1-4ec064b014fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50e6a4397dc84724864bc6f5f1151126', 'request_id': 'req-a26277ac-a5ad-448c-a4e1-4ec064b014fb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:52.539060 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a26277ac-a5ad-448c-a4e1-4ec064b014fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50e6a4397dc84724864bc6f5f1151126', 'request_id': 'req-a26277ac-a5ad-448c-a4e1-4ec064b014fb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:52.550402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a26277ac-a5ad-448c-a4e1-4ec064b014fb admin admin] The token cache is being invalidate because project 50e6a4397dc84724864bc6f5f1151126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:52.550687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a26277ac-a5ad-448c-a4e1-4ec064b014fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:52.551296 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a26277ac-a5ad-448c-a4e1-4ec064b014fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:52.552814 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1319/2685] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:52 2022] DELETE /identity/v3/projects/50e6a4397dc84724864bc6f5f1151126 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:52.556762 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e145980-85bc-4f64-a2b6-e24146e76b70 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:52.558269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2e145980-85bc-4f64-a2b6-e24146e76b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:52.571221 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e145980-85bc-4f64-a2b6-e24146e76b70 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:52.625907 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62f70a-304a-4d31-962a-60c9b2f8d937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:52.626811 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62f70a-304a-4d31-962a-60c9b2f8d937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:52.627349 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62f70a-304a-4d31-962a-60c9b2f8d937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:52.627626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62f70a-304a-4d31-962a-60c9b2f8d937 admin admin] PATH_INFO: `/v3/users/05a64f7b37694096a79b2822d1e023df` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:52.628810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62f70a-304a-4d31-962a-60c9b2f8d937 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05a64f7b37694096a79b2822d1e023df)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:52.630761 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62f70a-304a-4d31-962a-60c9b2f8d937 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:52.679117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ed62f70a-304a-4d31-962a-60c9b2f8d937 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05a64f7b37694096a79b2822d1e023df'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:52.679703 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ed62f70a-304a-4d31-962a-60c9b2f8d937 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05a64f7b37694096a79b2822d1e023df'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:52.680299 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ed62f70a-304a-4d31-962a-60c9b2f8d937 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05a64f7b37694096a79b2822d1e023df'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:52.692057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ed62f70a-304a-4d31-962a-60c9b2f8d937 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05a64f7b37694096a79b2822d1e023df'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:52.693475 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:52.701385 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ed62f70a-304a-4d31-962a-60c9b2f8d937 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05a64f7b37694096a79b2822d1e023df'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:52.714743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1367/2686] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:49:52 2022] DELETE /identity/v3/users/05a64f7b37694096a79b2822d1e023df => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:49:52.990684 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26277ac-a5ad-448c-a4e1-4ec064b014fb admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:49:52.992135 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a26277ac-a5ad-448c-a4e1-4ec064b014fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:49:53.005402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26277ac-a5ad-448c-a4e1-4ec064b014fb admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:49:53.061803 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061102f3-0465-46d4-8c1a-6079db319562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:49:53.062768 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061102f3-0465-46d4-8c1a-6079db319562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:49:53.063054 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061102f3-0465-46d4-8c1a-6079db319562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:49:53.063332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061102f3-0465-46d4-8c1a-6079db319562 admin admin] PATH_INFO: `/v3/projects/e5f0bca636ae47a9b4c0edbb38a0cdf0` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:49:53.064509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061102f3-0465-46d4-8c1a-6079db319562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5f0bca636ae47a9b4c0edbb38a0cdf0)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:49:53.066477 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061102f3-0465-46d4-8c1a-6079db319562 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:49:53.113626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-061102f3-0465-46d4-8c1a-6079db319562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5f0bca636ae47a9b4c0edbb38a0cdf0', 'request_id': 'req-061102f3-0465-46d4-8c1a-6079db319562'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:53.119023 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-061102f3-0465-46d4-8c1a-6079db319562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5f0bca636ae47a9b4c0edbb38a0cdf0', 'request_id': 'req-061102f3-0465-46d4-8c1a-6079db319562'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:53.126161 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-061102f3-0465-46d4-8c1a-6079db319562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5f0bca636ae47a9b4c0edbb38a0cdf0', 'request_id': 'req-061102f3-0465-46d4-8c1a-6079db319562'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:53.134181 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:49:53.139029 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-061102f3-0465-46d4-8c1a-6079db319562 admin admin] The token cache is being invalidate because project e5f0bca636ae47a9b4c0edbb38a0cdf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:49:53.139525 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-061102f3-0465-46d4-8c1a-6079db319562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:53.140270 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-061102f3-0465-46d4-8c1a-6079db319562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:49:53.142295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1320/2687] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:49:52 2022] DELETE /identity/v3/projects/e5f0bca636ae47a9b4c0edbb38a0cdf0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:01.932347 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62f70a-304a-4d31-962a-60c9b2f8d937 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:01.936346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ed62f70a-304a-4d31-962a-60c9b2f8d937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:02.021097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62f70a-304a-4d31-962a-60c9b2f8d937 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:02.087479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acde87-575a-49ff-b318-ae4c7261e662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:02.087479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acde87-575a-49ff-b318-ae4c7261e662 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:02.087479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acde87-575a-49ff-b318-ae4c7261e662 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:02.087479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acde87-575a-49ff-b318-ae4c7261e662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:02.088480 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f5acde87-575a-49ff-b318-ae4c7261e662 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:02.161649 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acde87-575a-49ff-b318-ae4c7261e662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:02.166247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acde87-575a-49ff-b318-ae4c7261e662 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:02.179781 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1368/2688] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:04.511799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061102f3-0465-46d4-8c1a-6079db319562 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:04.513415 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-061102f3-0465-46d4-8c1a-6079db319562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:04.584623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061102f3-0465-46d4-8c1a-6079db319562 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:04.590105 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42feef26-a733-41d8-8f25-b3261aadca12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:04.591097 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42feef26-a733-41d8-8f25-b3261aadca12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:04.591436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42feef26-a733-41d8-8f25-b3261aadca12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:04.591769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42feef26-a733-41d8-8f25-b3261aadca12 admin admin] PATH_INFO: `/v3/users/0b82d36d9dbd49dfb87f4aa5537f48bf` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:04.593017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42feef26-a733-41d8-8f25-b3261aadca12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b82d36d9dbd49dfb87f4aa5537f48bf)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:04.595300 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42feef26-a733-41d8-8f25-b3261aadca12 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:04.641794 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-42feef26-a733-41d8-8f25-b3261aadca12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b82d36d9dbd49dfb87f4aa5537f48bf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:04.654017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-42feef26-a733-41d8-8f25-b3261aadca12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b82d36d9dbd49dfb87f4aa5537f48bf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:04.660868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-42feef26-a733-41d8-8f25-b3261aadca12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b82d36d9dbd49dfb87f4aa5537f48bf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:04.661550 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-42feef26-a733-41d8-8f25-b3261aadca12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b82d36d9dbd49dfb87f4aa5537f48bf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:04.671725 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-42feef26-a733-41d8-8f25-b3261aadca12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b82d36d9dbd49dfb87f4aa5537f48bf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:04.678610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1321/2689] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:50:04 2022] DELETE /identity/v3/users/0b82d36d9dbd49dfb87f4aa5537f48bf => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:04.683112 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ef5e6-ef1b-4047-bc3a-62599f69b40a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:04.683606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ef5e6-ef1b-4047-bc3a-62599f69b40a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:04.683770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ef5e6-ef1b-4047-bc3a-62599f69b40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:04.710608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-814ef5e6-ef1b-4047-bc3a-62599f69b40a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:04.711866 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-814ef5e6-ef1b-4047-bc3a-62599f69b40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:04.731452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:04.782837 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1369/2690] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:04.792718 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42feef26-a733-41d8-8f25-b3261aadca12 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:04.795247 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-42feef26-a733-41d8-8f25-b3261aadca12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:04.858013 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ef5e6-ef1b-4047-bc3a-62599f69b40a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:04.859463 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-814ef5e6-ef1b-4047-bc3a-62599f69b40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:04.867362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42feef26-a733-41d8-8f25-b3261aadca12 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:04.872053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ecefb-7804-4fe8-a5c4-70844a60c30a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:04.872796 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ecefb-7804-4fe8-a5c4-70844a60c30a service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:04.873296 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ecefb-7804-4fe8-a5c4-70844a60c30a service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:04.873641 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ecefb-7804-4fe8-a5c4-70844a60c30a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:04.888028 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ecefb-7804-4fe8-a5c4-70844a60c30a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:04.889603 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ecefb-7804-4fe8-a5c4-70844a60c30a service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:04.901479 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1322/2691] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:04.987854 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ef5e6-ef1b-4047-bc3a-62599f69b40a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:04.996180 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5c9ae-e16b-4988-9278-4c004c462f05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1308deea1ea4629a1ac22d4b903ad4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:04.997187 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5c9ae-e16b-4988-9278-4c004c462f05 service placement] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:04.997713 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5c9ae-e16b-4988-9278-4c004c462f05 service placement] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:04.998239 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5c9ae-e16b-4988-9278-4c004c462f05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:05.001041 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-22f5c9ae-e16b-4988-9278-4c004c462f05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:05.015769 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5c9ae-e16b-4988-9278-4c004c462f05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:05.017711 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5c9ae-e16b-4988-9278-4c004c462f05 service placement] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:05.043502 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1370/2692] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:50:05.270130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ecefb-7804-4fe8-a5c4-70844a60c30a service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:05.271928 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7c3ecefb-7804-4fe8-a5c4-70844a60c30a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:05.292446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ecefb-7804-4fe8-a5c4-70844a60c30a service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:05.298298 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:05.299185 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:05.299468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:05.299867 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43 admin admin] PATH_INFO: `/v3/projects/b6e08a151f2c4a0db9366ea417c722c1` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:05.302086 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6e08a151f2c4a0db9366ea417c722c1)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:05.304398 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:05.327964 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:05.365977 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6e08a151f2c4a0db9366ea417c722c1', 'request_id': 'req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:05.373171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6e08a151f2c4a0db9366ea417c722c1', 'request_id': 'req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:05.382714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6e08a151f2c4a0db9366ea417c722c1', 'request_id': 'req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:05.395849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43 admin admin] The token cache is being invalidate because project b6e08a151f2c4a0db9366ea417c722c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:50:05.396162 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:05.396853 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:05.398640 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1323/2693] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:50:05 2022] DELETE /identity/v3/projects/b6e08a151f2c4a0db9366ea417c722c1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:05.402558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5c9ae-e16b-4988-9278-4c004c462f05 service placement] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:05.412992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5c9ae-e16b-4988-9278-4c004c462f05 service placement] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:05.470407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a1178-e373-4c9f-a3d8-83877cc16987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:05.471289 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a1178-e373-4c9f-a3d8-83877cc16987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:05.471617 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a1178-e373-4c9f-a3d8-83877cc16987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:05.471984 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a1178-e373-4c9f-a3d8-83877cc16987 admin admin] PATH_INFO: `/v3/users/a812309492f24b0fb9ea196515c5d8c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:05.473207 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a1178-e373-4c9f-a3d8-83877cc16987 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a812309492f24b0fb9ea196515c5d8c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:05.475367 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a1178-e373-4c9f-a3d8-83877cc16987 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:05.534247 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:05.552386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-388a1178-e373-4c9f-a3d8-83877cc16987 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a812309492f24b0fb9ea196515c5d8c8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:05.552960 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-388a1178-e373-4c9f-a3d8-83877cc16987 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a812309492f24b0fb9ea196515c5d8c8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:05.553557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-388a1178-e373-4c9f-a3d8-83877cc16987 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a812309492f24b0fb9ea196515c5d8c8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:05.566849 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-388a1178-e373-4c9f-a3d8-83877cc16987 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a812309492f24b0fb9ea196515c5d8c8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:05.574078 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-388a1178-e373-4c9f-a3d8-83877cc16987 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a812309492f24b0fb9ea196515c5d8c8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:05.588370 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1371/2694] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:50:05 2022] DELETE /identity/v3/users/a812309492f24b0fb9ea196515c5d8c8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:05.898285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:05.899738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:05.922443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed65fc-293d-4ba6-8bdf-f415cdc5ad43 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:05.994598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb603d-331c-4cde-a83a-0aa1d4b8ad15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:05.995476 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb603d-331c-4cde-a83a-0aa1d4b8ad15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:05.995760 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb603d-331c-4cde-a83a-0aa1d4b8ad15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:05.996053 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb603d-331c-4cde-a83a-0aa1d4b8ad15 admin admin] PATH_INFO: `/v3/projects/d7d928ee6953437e97d8d265e32498ee` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:05.997232 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb603d-331c-4cde-a83a-0aa1d4b8ad15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7d928ee6953437e97d8d265e32498ee)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:05.999174 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb603d-331c-4cde-a83a-0aa1d4b8ad15 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:06.056727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ceeb603d-331c-4cde-a83a-0aa1d4b8ad15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7d928ee6953437e97d8d265e32498ee', 'request_id': 'req-ceeb603d-331c-4cde-a83a-0aa1d4b8ad15'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:06.064712 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ceeb603d-331c-4cde-a83a-0aa1d4b8ad15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7d928ee6953437e97d8d265e32498ee', 'request_id': 'req-ceeb603d-331c-4cde-a83a-0aa1d4b8ad15'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:06.074364 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ceeb603d-331c-4cde-a83a-0aa1d4b8ad15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7d928ee6953437e97d8d265e32498ee', 'request_id': 'req-ceeb603d-331c-4cde-a83a-0aa1d4b8ad15'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:06.087073 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ceeb603d-331c-4cde-a83a-0aa1d4b8ad15 admin admin] The token cache is being invalidate because project d7d928ee6953437e97d8d265e32498ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:50:06.087313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ceeb603d-331c-4cde-a83a-0aa1d4b8ad15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:06.088589 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ceeb603d-331c-4cde-a83a-0aa1d4b8ad15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:06.090911 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1324/2695] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:50:05 2022] DELETE /identity/v3/projects/d7d928ee6953437e97d8d265e32498ee => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:06.095931 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a881027-7fe1-4a07-abbf-cb7b748ead54 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:06.096406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a881027-7fe1-4a07-abbf-cb7b748ead54 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:06.096620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a881027-7fe1-4a07-abbf-cb7b748ead54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:06.150692 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4a881027-7fe1-4a07-abbf-cb7b748ead54 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:06.151775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4a881027-7fe1-4a07-abbf-cb7b748ead54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:06.222068 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1372/2696] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:06.227116 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe07f02-ec93-4fb1-86a2-e38ff3a8df3c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:06.227614 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe07f02-ec93-4fb1-86a2-e38ff3a8df3c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:06.227961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe07f02-ec93-4fb1-86a2-e38ff3a8df3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:06.255541 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-efe07f02-ec93-4fb1-86a2-e38ff3a8df3c None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:06.256761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-efe07f02-ec93-4fb1-86a2-e38ff3a8df3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:06.271178 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a881027-7fe1-4a07-abbf-cb7b748ead54 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:06.282474 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1325/2697] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:06.282831 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a881027-7fe1-4a07-abbf-cb7b748ead54 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:06.288243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24042f7e-daac-41f0-bff6-ec09f98df4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:06.288990 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24042f7e-daac-41f0-bff6-ec09f98df4a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:06.289201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24042f7e-daac-41f0-bff6-ec09f98df4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:06.289464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24042f7e-daac-41f0-bff6-ec09f98df4a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:06.290382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24042f7e-daac-41f0-bff6-ec09f98df4a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:50:06.290735 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24042f7e-daac-41f0-bff6-ec09f98df4a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:06.292360 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24042f7e-daac-41f0-bff6-ec09f98df4a3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:06.305934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1373/2698] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:50:06.310399 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe07f02-ec93-4fb1-86a2-e38ff3a8df3c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:06.320256 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe07f02-ec93-4fb1-86a2-e38ff3a8df3c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:06.326027 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b90509-c5c1-49be-bee6-e3e15c9229df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:06.326704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b90509-c5c1-49be-bee6-e3e15c9229df admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:06.326988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b90509-c5c1-49be-bee6-e3e15c9229df admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:06.327263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b90509-c5c1-49be-bee6-e3e15c9229df admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:06.327983 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b90509-c5c1-49be-bee6-e3e15c9229df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:06.329909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b90509-c5c1-49be-bee6-e3e15c9229df admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:06.364828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1326/2699] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:50:06 2022] POST /identity/v3/projects => generated 407 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:50:06.366170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24042f7e-daac-41f0-bff6-ec09f98df4a3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:06.376955 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24042f7e-daac-41f0-bff6-ec09f98df4a3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:06.447449 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc00a84-f911-49cb-aacd-b4c1ef2a6764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:06.448187 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc00a84-f911-49cb-aacd-b4c1ef2a6764 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:06.448467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc00a84-f911-49cb-aacd-b4c1ef2a6764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:06.448744 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc00a84-f911-49cb-aacd-b4c1ef2a6764 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:06.449434 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc00a84-f911-49cb-aacd-b4c1ef2a6764 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:06.451418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc00a84-f911-49cb-aacd-b4c1ef2a6764 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:06.477070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1374/2700] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:50:06 2022] POST /identity/v3/users => generated 304 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:06.478572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b90509-c5c1-49be-bee6-e3e15c9229df admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:06.493009 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b90509-c5c1-49be-bee6-e3e15c9229df admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:06.500496 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5a90f-91e4-4f4d-a61e-21926e04a84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:06.501624 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5a90f-91e4-4f4d-a61e-21926e04a84e admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:06.502420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5a90f-91e4-4f4d-a61e-21926e04a84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:06.502935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5a90f-91e4-4f4d-a61e-21926e04a84e admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:06.504314 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5a90f-91e4-4f4d-a61e-21926e04a84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:06.507411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5a90f-91e4-4f4d-a61e-21926e04a84e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:06.510713 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:06.523346 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1327/2701] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:06.528009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc00a84-f911-49cb-aacd-b4c1ef2a6764 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:06.538161 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc00a84-f911-49cb-aacd-b4c1ef2a6764 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:06.542857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2d6d4-b6af-4b05-9b02-8df43ac517ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:06.543516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2d6d4-b6af-4b05-9b02-8df43ac517ac admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:06.543806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2d6d4-b6af-4b05-9b02-8df43ac517ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:06.544084 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2d6d4-b6af-4b05-9b02-8df43ac517ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:06.544812 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2d6d4-b6af-4b05-9b02-8df43ac517ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:06.546451 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2d6d4-b6af-4b05-9b02-8df43ac517ac admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:06.547734 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:06.559684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1375/2702] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:06.563578 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5a90f-91e4-4f4d-a61e-21926e04a84e admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:06.574216 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5a90f-91e4-4f4d-a61e-21926e04a84e admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:06.578841 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbee506-ba99-4496-aab3-cc0c10d453fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:06.579719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbee506-ba99-4496-aab3-cc0c10d453fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:06.579999 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbee506-ba99-4496-aab3-cc0c10d453fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:06.580302 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbee506-ba99-4496-aab3-cc0c10d453fe admin admin] PATH_INFO: `/v3/projects/ed94a9a3ae8e4ceebb36fa398082fbe1/users/264ef3b57c5b4b7784a023a6fa0a53b9/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:06.595694 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbee506-ba99-4496-aab3-cc0c10d453fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed94a9a3ae8e4ceebb36fa398082fbe1, user_id=264ef3b57c5b4b7784a023a6fa0a53b9, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:06.598373 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbee506-ba99-4496-aab3-cc0c10d453fe admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:06.615725 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1328/2703] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:50:06 2022] PUT /identity/v3/projects/ed94a9a3ae8e4ceebb36fa398082fbe1/users/264ef3b57c5b4b7784a023a6fa0a53b9/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:06.625513 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcfa87-8623-4066-a3f2-de2f48e0dcc6 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:06.625805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcfa87-8623-4066-a3f2-de2f48e0dcc6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:06.626117 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcfa87-8623-4066-a3f2-de2f48e0dcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:06.657683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-31fcfa87-8623-4066-a3f2-de2f48e0dcc6 None None] MFA Rules not processed for user `264ef3b57c5b4b7784a023a6fa0a53b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:06.659203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-31fcfa87-8623-4066-a3f2-de2f48e0dcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:06.732690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1376/2704] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:06 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:06.737263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbee506-ba99-4496-aab3-cc0c10d453fe admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:06.832990 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbee506-ba99-4496-aab3-cc0c10d453fe admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:06.838091 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d6b84-76ff-41c3-a04e-c5eb017630ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:06.838798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d6b84-76ff-41c3-a04e-c5eb017630ea admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:06.839235 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d6b84-76ff-41c3-a04e-c5eb017630ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:06.839511 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d6b84-76ff-41c3-a04e-c5eb017630ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:06.846142 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d6b84-76ff-41c3-a04e-c5eb017630ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:06.848716 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d6b84-76ff-41c3-a04e-c5eb017630ea admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:06.895844 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1329/2705] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:50:06 2022] POST /identity/v3/projects => generated 407 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:06.897270 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fcfa87-8623-4066-a3f2-de2f48e0dcc6 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:06.908434 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fcfa87-8623-4066-a3f2-de2f48e0dcc6 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:06.973605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5884a-4e13-481f-9565-1ebf79d3f514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:06.974329 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5884a-4e13-481f-9565-1ebf79d3f514 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:06.974622 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5884a-4e13-481f-9565-1ebf79d3f514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:06.974904 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5884a-4e13-481f-9565-1ebf79d3f514 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:06.975605 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5884a-4e13-481f-9565-1ebf79d3f514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:06.977637 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5884a-4e13-481f-9565-1ebf79d3f514 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:07.003461 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1377/2706] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:50:06 2022] POST /identity/v3/users => generated 310 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:07.004629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d6b84-76ff-41c3-a04e-c5eb017630ea admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:07.018866 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d6b84-76ff-41c3-a04e-c5eb017630ea admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:07.025107 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76e2de-34b2-4878-8d8d-3e60b091d095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:07.026124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76e2de-34b2-4878-8d8d-3e60b091d095 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:07.026516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76e2de-34b2-4878-8d8d-3e60b091d095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:07.027006 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76e2de-34b2-4878-8d8d-3e60b091d095 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:07.028019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76e2de-34b2-4878-8d8d-3e60b091d095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:07.030196 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76e2de-34b2-4878-8d8d-3e60b091d095 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:07.043246 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1330/2707] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:07.047276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5884a-4e13-481f-9565-1ebf79d3f514 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:07.061730 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5884a-4e13-481f-9565-1ebf79d3f514 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:07.069361 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c73d3a-9ee0-4311-9f95-5dc71874471d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:07.070560 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c73d3a-9ee0-4311-9f95-5dc71874471d admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:07.071025 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c73d3a-9ee0-4311-9f95-5dc71874471d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:07.077931 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c73d3a-9ee0-4311-9f95-5dc71874471d admin admin] PATH_INFO: `/v3/projects/720c61b0410f4a5e905efc6e29b8951f/users/5eaaa2f595a74f84b57c36c346548cb8/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:07.096968 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c73d3a-9ee0-4311-9f95-5dc71874471d admin admin] RBAC: Authorizing `identity:create_grant(project_id=720c61b0410f4a5e905efc6e29b8951f, user_id=5eaaa2f595a74f84b57c36c346548cb8, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:07.101047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c73d3a-9ee0-4311-9f95-5dc71874471d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:07.117800 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1378/2708] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:50:07 2022] PUT /identity/v3/projects/720c61b0410f4a5e905efc6e29b8951f/users/5eaaa2f595a74f84b57c36c346548cb8/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:50:07.122380 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76e2de-34b2-4878-8d8d-3e60b091d095 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:07.134446 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76e2de-34b2-4878-8d8d-3e60b091d095 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:07.200559 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fddb7f-5529-494c-9982-f6a97432d846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:07.201358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fddb7f-5529-494c-9982-f6a97432d846 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:07.201790 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fddb7f-5529-494c-9982-f6a97432d846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:07.202081 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fddb7f-5529-494c-9982-f6a97432d846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:07.202876 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fddb7f-5529-494c-9982-f6a97432d846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:07.204491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fddb7f-5529-494c-9982-f6a97432d846 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:07.216761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1331/2709] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:50:07.220671 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c73d3a-9ee0-4311-9f95-5dc71874471d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:07.232543 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c73d3a-9ee0-4311-9f95-5dc71874471d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:07.237804 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70257d09-e1e2-4271-89db-857920a5b2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:07.238581 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70257d09-e1e2-4271-89db-857920a5b2ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:07.238867 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70257d09-e1e2-4271-89db-857920a5b2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:07.239147 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70257d09-e1e2-4271-89db-857920a5b2ef admin admin] PATH_INFO: `/v3/projects/720c61b0410f4a5e905efc6e29b8951f/users/5eaaa2f595a74f84b57c36c346548cb8/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:07.241317 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70257d09-e1e2-4271-89db-857920a5b2ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=720c61b0410f4a5e905efc6e29b8951f, user_id=5eaaa2f595a74f84b57c36c346548cb8, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:07.244844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70257d09-e1e2-4271-89db-857920a5b2ef admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:07.257633 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1379/2710] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:50:07 2022] PUT /identity/v3/projects/720c61b0410f4a5e905efc6e29b8951f/users/5eaaa2f595a74f84b57c36c346548cb8/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:07.325491 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f59ccf-aa8b-4548-af04-5ac2b55aabb2 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:07.325849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f59ccf-aa8b-4548-af04-5ac2b55aabb2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:07.326134 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f59ccf-aa8b-4548-af04-5ac2b55aabb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:07.356978 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-92f59ccf-aa8b-4548-af04-5ac2b55aabb2 None None] MFA Rules not processed for user `5eaaa2f595a74f84b57c36c346548cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:07.358262 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-92f59ccf-aa8b-4548-af04-5ac2b55aabb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:07.395274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:07.448056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1332/2711] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:07 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:07.457475 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70257d09-e1e2-4271-89db-857920a5b2ef admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:07.459341 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-70257d09-e1e2-4271-89db-857920a5b2ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:07.518670 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:07.539936 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70257d09-e1e2-4271-89db-857920a5b2ef admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:07.545106 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66bf94f-7bdf-4a93-9cc8-d0d83082d249 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:07.545495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66bf94f-7bdf-4a93-9cc8-d0d83082d249 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:07.545782 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66bf94f-7bdf-4a93-9cc8-d0d83082d249 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:07.546100 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66bf94f-7bdf-4a93-9cc8-d0d83082d249 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:07.556281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66bf94f-7bdf-4a93-9cc8-d0d83082d249 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:07.557850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66bf94f-7bdf-4a93-9cc8-d0d83082d249 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:07.573216 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1380/2712] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:07 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:07.745861 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f59ccf-aa8b-4548-af04-5ac2b55aabb2 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:07.747302 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-92f59ccf-aa8b-4548-af04-5ac2b55aabb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:07.760385 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f59ccf-aa8b-4548-af04-5ac2b55aabb2 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:07.764376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fdef0-f028-48d9-a964-0efd2a4dd1b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:07.765179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fdef0-f028-48d9-a964-0efd2a4dd1b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:07.765478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fdef0-f028-48d9-a964-0efd2a4dd1b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:07.765753 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fdef0-f028-48d9-a964-0efd2a4dd1b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:07.825870 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fdef0-f028-48d9-a964-0efd2a4dd1b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:07.827397 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fdef0-f028-48d9-a964-0efd2a4dd1b8 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:07.838716 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1333/2713] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:07 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:10.010181 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66bf94f-7bdf-4a93-9cc8-d0d83082d249 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:10.011582 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d66bf94f-7bdf-4a93-9cc8-d0d83082d249 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:10.087568 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66bf94f-7bdf-4a93-9cc8-d0d83082d249 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:10.091179 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a435fc4-f318-4fb5-88c2-6a84be15676d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:10.091813 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a435fc4-f318-4fb5-88c2-6a84be15676d service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:10.092098 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a435fc4-f318-4fb5-88c2-6a84be15676d service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:10.092374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a435fc4-f318-4fb5-88c2-6a84be15676d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:10.094152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0a435fc4-f318-4fb5-88c2-6a84be15676d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:10.154812 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a435fc4-f318-4fb5-88c2-6a84be15676d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:10.156233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a435fc4-f318-4fb5-88c2-6a84be15676d service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:10.166276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1381/2714] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:13.062718 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fdef0-f028-48d9-a964-0efd2a4dd1b8 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:13.064409 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a85fdef0-f028-48d9-a964-0efd2a4dd1b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:13.083424 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fdef0-f028-48d9-a964-0efd2a4dd1b8 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:13.088816 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32add35e-c159-42eb-9653-64f939bf13be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:13.089689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32add35e-c159-42eb-9653-64f939bf13be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:13.090034 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32add35e-c159-42eb-9653-64f939bf13be admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:13.090290 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32add35e-c159-42eb-9653-64f939bf13be admin admin] PATH_INFO: `/v3/users/d9e5bc258de54fd6b3811401783a7408` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:13.091538 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32add35e-c159-42eb-9653-64f939bf13be admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9e5bc258de54fd6b3811401783a7408)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:13.093478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32add35e-c159-42eb-9653-64f939bf13be admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:13.145751 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-32add35e-c159-42eb-9653-64f939bf13be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9e5bc258de54fd6b3811401783a7408'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:13.158887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-32add35e-c159-42eb-9653-64f939bf13be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9e5bc258de54fd6b3811401783a7408'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:13.165896 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-32add35e-c159-42eb-9653-64f939bf13be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9e5bc258de54fd6b3811401783a7408'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:13.166710 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-32add35e-c159-42eb-9653-64f939bf13be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9e5bc258de54fd6b3811401783a7408'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:13.175971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-32add35e-c159-42eb-9653-64f939bf13be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9e5bc258de54fd6b3811401783a7408'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:13.187568 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1334/2715] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:50:13 2022] DELETE /identity/v3/users/d9e5bc258de54fd6b3811401783a7408 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:13.353346 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a435fc4-f318-4fb5-88c2-6a84be15676d service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:13.363620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a435fc4-f318-4fb5-88c2-6a84be15676d service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:13.420537 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aba1c3-f952-4f3f-88da-c303ee3e5ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:13.421318 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aba1c3-f952-4f3f-88da-c303ee3e5ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:13.421623 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aba1c3-f952-4f3f-88da-c303ee3e5ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:13.421882 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aba1c3-f952-4f3f-88da-c303ee3e5ab0 admin admin] PATH_INFO: `/v3/projects/e54281ff11484f77b0d20fd039ef938e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:13.423100 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aba1c3-f952-4f3f-88da-c303ee3e5ab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e54281ff11484f77b0d20fd039ef938e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:13.425020 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aba1c3-f952-4f3f-88da-c303ee3e5ab0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:13.436644 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:13.471548 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f7aba1c3-f952-4f3f-88da-c303ee3e5ab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e54281ff11484f77b0d20fd039ef938e', 'request_id': 'req-f7aba1c3-f952-4f3f-88da-c303ee3e5ab0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:13.478707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f7aba1c3-f952-4f3f-88da-c303ee3e5ab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e54281ff11484f77b0d20fd039ef938e', 'request_id': 'req-f7aba1c3-f952-4f3f-88da-c303ee3e5ab0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:13.484750 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f7aba1c3-f952-4f3f-88da-c303ee3e5ab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e54281ff11484f77b0d20fd039ef938e', 'request_id': 'req-f7aba1c3-f952-4f3f-88da-c303ee3e5ab0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:13.495662 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f7aba1c3-f952-4f3f-88da-c303ee3e5ab0 admin admin] The token cache is being invalidate because project e54281ff11484f77b0d20fd039ef938e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:50:13.496001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f7aba1c3-f952-4f3f-88da-c303ee3e5ab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:13.496626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f7aba1c3-f952-4f3f-88da-c303ee3e5ab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:13.498169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1382/2716] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:50:13 2022] DELETE /identity/v3/projects/e54281ff11484f77b0d20fd039ef938e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:13.503382 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675c735-cea1-4c71-869a-facb90d4a4a1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:13.503674 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675c735-cea1-4c71-869a-facb90d4a4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:13.503953 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675c735-cea1-4c71-869a-facb90d4a4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:13.521623 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8675c735-cea1-4c71-869a-facb90d4a4a1 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:13.522858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8675c735-cea1-4c71-869a-facb90d4a4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:13.554364 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:13.583955 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1335/2717] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:13.591535 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848edaf1-6d4a-4feb-9c92-02e50b3fa90d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:13.591853 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848edaf1-6d4a-4feb-9c92-02e50b3fa90d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:13.592127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848edaf1-6d4a-4feb-9c92-02e50b3fa90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:13.610073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-848edaf1-6d4a-4feb-9c92-02e50b3fa90d None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:13.611274 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-848edaf1-6d4a-4feb-9c92-02e50b3fa90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:13.619597 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1383/2718] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:50:13.623668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675c735-cea1-4c71-869a-facb90d4a4a1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:13.633569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675c735-cea1-4c71-869a-facb90d4a4a1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:13.638771 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb40080-1315-446d-86b5-5e53633045e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:13.639420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb40080-1315-446d-86b5-5e53633045e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:13.639705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb40080-1315-446d-86b5-5e53633045e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:13.639985 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb40080-1315-446d-86b5-5e53633045e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:13.640713 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb40080-1315-446d-86b5-5e53633045e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:50:13.641021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb40080-1315-446d-86b5-5e53633045e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:13.642669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb40080-1315-446d-86b5-5e53633045e1 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:13.654194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1336/2719] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:50:13.658282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848edaf1-6d4a-4feb-9c92-02e50b3fa90d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:13.667721 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848edaf1-6d4a-4feb-9c92-02e50b3fa90d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:13.672680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19dd254-9d94-4a65-befd-71b08b980d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:13.673299 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19dd254-9d94-4a65-befd-71b08b980d9a admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:13.673587 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19dd254-9d94-4a65-befd-71b08b980d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:13.673864 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19dd254-9d94-4a65-befd-71b08b980d9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:13.674583 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19dd254-9d94-4a65-befd-71b08b980d9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:13.676482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19dd254-9d94-4a65-befd-71b08b980d9a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:13.702146 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1384/2720] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:50:13 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:13.706195 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb40080-1315-446d-86b5-5e53633045e1 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:13.716014 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb40080-1315-446d-86b5-5e53633045e1 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:13.771072 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ac8cd-8fce-4ef0-8422-aed3c58a41c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:13.771721 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ac8cd-8fce-4ef0-8422-aed3c58a41c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:13.772006 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ac8cd-8fce-4ef0-8422-aed3c58a41c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:13.772282 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ac8cd-8fce-4ef0-8422-aed3c58a41c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:13.772968 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ac8cd-8fce-4ef0-8422-aed3c58a41c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:13.775540 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ac8cd-8fce-4ef0-8422-aed3c58a41c6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:13.795157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1337/2721] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:50:13 2022] POST /identity/v3/users => generated 291 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:50:13.799175 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19dd254-9d94-4a65-befd-71b08b980d9a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:13.810207 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19dd254-9d94-4a65-befd-71b08b980d9a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:13.814852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97271043-b25f-4be1-b043-0f287e337367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:13.815511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97271043-b25f-4be1-b043-0f287e337367 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:13.815797 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97271043-b25f-4be1-b043-0f287e337367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:13.816093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97271043-b25f-4be1-b043-0f287e337367 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:13.816818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97271043-b25f-4be1-b043-0f287e337367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:13.818493 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97271043-b25f-4be1-b043-0f287e337367 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:13.828661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1385/2722] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:13.832303 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ac8cd-8fce-4ef0-8422-aed3c58a41c6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:13.842068 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ac8cd-8fce-4ef0-8422-aed3c58a41c6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:13.846777 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d24bd2-f954-40ec-a4d5-b8b77ff66cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:13.847436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d24bd2-f954-40ec-a4d5-b8b77ff66cbb admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:13.847778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d24bd2-f954-40ec-a4d5-b8b77ff66cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:13.847988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d24bd2-f954-40ec-a4d5-b8b77ff66cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:13.848722 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d24bd2-f954-40ec-a4d5-b8b77ff66cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:13.850368 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d24bd2-f954-40ec-a4d5-b8b77ff66cbb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:13.861411 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1338/2723] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:13.865284 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97271043-b25f-4be1-b043-0f287e337367 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:13.876222 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97271043-b25f-4be1-b043-0f287e337367 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:13.883353 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc860f-875c-45ce-acbd-389000110537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:13.884485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc860f-875c-45ce-acbd-389000110537 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:13.884834 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc860f-875c-45ce-acbd-389000110537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:13.885405 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc860f-875c-45ce-acbd-389000110537 admin admin] PATH_INFO: `/v3/projects/6b42ffcbb47a464d8760cff525755b4e/users/84c436bf88094a568a9a3e7ed761c4fa/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:13.900264 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc860f-875c-45ce-acbd-389000110537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b42ffcbb47a464d8760cff525755b4e, user_id=84c436bf88094a568a9a3e7ed761c4fa, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:13.903039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc860f-875c-45ce-acbd-389000110537 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:13.917050 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1386/2724] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:50:13 2022] PUT /identity/v3/projects/6b42ffcbb47a464d8760cff525755b4e/users/84c436bf88094a568a9a3e7ed761c4fa/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:13.927889 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbb850-eba4-4d7b-8827-bd6c6093a3e2 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:13.928117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbb850-eba4-4d7b-8827-bd6c6093a3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:13.928395 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbb850-eba4-4d7b-8827-bd6c6093a3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:13.955754 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-40dbb850-eba4-4d7b-8827-bd6c6093a3e2 None None] MFA Rules not processed for user `84c436bf88094a568a9a3e7ed761c4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:13.956961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-40dbb850-eba4-4d7b-8827-bd6c6093a3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:14.030516 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1339/2725] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:50:14.042540 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc860f-875c-45ce-acbd-389000110537 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:14.044292 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1afc860f-875c-45ce-acbd-389000110537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:14.151091 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc860f-875c-45ce-acbd-389000110537 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:14.157010 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd675407-8092-4be3-9e08-6b9c10bc5ead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:14.157650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd675407-8092-4be3-9e08-6b9c10bc5ead service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:14.157972 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd675407-8092-4be3-9e08-6b9c10bc5ead service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:14.158253 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd675407-8092-4be3-9e08-6b9c10bc5ead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:14.164206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:14.167200 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd675407-8092-4be3-9e08-6b9c10bc5ead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:14.169191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd675407-8092-4be3-9e08-6b9c10bc5ead service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:14.179944 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1387/2726] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:15.250828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbb850-eba4-4d7b-8827-bd6c6093a3e2 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:15.252489 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-40dbb850-eba4-4d7b-8827-bd6c6093a3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:15.286761 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:15.322252 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbb850-eba4-4d7b-8827-bd6c6093a3e2 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:15.326799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39ed48-e3ec-432f-8592-054a280d6c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:15.327669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39ed48-e3ec-432f-8592-054a280d6c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:15.327950 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39ed48-e3ec-432f-8592-054a280d6c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:15.328222 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39ed48-e3ec-432f-8592-054a280d6c24 admin admin] PATH_INFO: `/v3/users/264ef3b57c5b4b7784a023a6fa0a53b9` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:15.329399 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39ed48-e3ec-432f-8592-054a280d6c24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=264ef3b57c5b4b7784a023a6fa0a53b9)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:15.331341 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39ed48-e3ec-432f-8592-054a280d6c24 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:15.378812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-bb39ed48-e3ec-432f-8592-054a280d6c24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '264ef3b57c5b4b7784a023a6fa0a53b9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:15.392864 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-bb39ed48-e3ec-432f-8592-054a280d6c24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '264ef3b57c5b4b7784a023a6fa0a53b9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:15.400271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-bb39ed48-e3ec-432f-8592-054a280d6c24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '264ef3b57c5b4b7784a023a6fa0a53b9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:15.401028 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-bb39ed48-e3ec-432f-8592-054a280d6c24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '264ef3b57c5b4b7784a023a6fa0a53b9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:15.409336 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-bb39ed48-e3ec-432f-8592-054a280d6c24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '264ef3b57c5b4b7784a023a6fa0a53b9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:15.416602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1340/2727] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:50:15 2022] DELETE /identity/v3/users/264ef3b57c5b4b7784a023a6fa0a53b9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:15.421199 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b76a4-bd6a-4c87-ac3d-5059afc20307 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:15.421764 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b76a4-bd6a-4c87-ac3d-5059afc20307 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:15.421874 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b76a4-bd6a-4c87-ac3d-5059afc20307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:15.439606 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9a6b76a4-bd6a-4c87-ac3d-5059afc20307 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:15.440803 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9a6b76a4-bd6a-4c87-ac3d-5059afc20307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:15.499302 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1388/2728] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:15.509413 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39ed48-e3ec-432f-8592-054a280d6c24 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:15.510852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bb39ed48-e3ec-432f-8592-054a280d6c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:15.568793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39ed48-e3ec-432f-8592-054a280d6c24 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:15.572621 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d2326-7e77-4b17-8a6f-8b6b0f3b0093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:15.573274 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d2326-7e77-4b17-8a6f-8b6b0f3b0093 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:15.573558 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d2326-7e77-4b17-8a6f-8b6b0f3b0093 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:15.573835 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d2326-7e77-4b17-8a6f-8b6b0f3b0093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:15.583950 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d2326-7e77-4b17-8a6f-8b6b0f3b0093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:15.585381 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d2326-7e77-4b17-8a6f-8b6b0f3b0093 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:15.595606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1341/2729] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:50:15.919658 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b76a4-bd6a-4c87-ac3d-5059afc20307 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:15.922193 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9a6b76a4-bd6a-4c87-ac3d-5059afc20307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:15.935218 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b76a4-bd6a-4c87-ac3d-5059afc20307 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:15.942666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e198fb6-04b1-4ec4-b5e5-e401772d5543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:15.944322 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e198fb6-04b1-4ec4-b5e5-e401772d5543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:15.944322 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e198fb6-04b1-4ec4-b5e5-e401772d5543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:15.945550 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e198fb6-04b1-4ec4-b5e5-e401772d5543 admin admin] PATH_INFO: `/v3/projects/ed94a9a3ae8e4ceebb36fa398082fbe1` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:15.945710 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e198fb6-04b1-4ec4-b5e5-e401772d5543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed94a9a3ae8e4ceebb36fa398082fbe1)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:15.947644 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e198fb6-04b1-4ec4-b5e5-e401772d5543 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:16.000272 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7e198fb6-04b1-4ec4-b5e5-e401772d5543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed94a9a3ae8e4ceebb36fa398082fbe1', 'request_id': 'req-7e198fb6-04b1-4ec4-b5e5-e401772d5543'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:16.007850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7e198fb6-04b1-4ec4-b5e5-e401772d5543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed94a9a3ae8e4ceebb36fa398082fbe1', 'request_id': 'req-7e198fb6-04b1-4ec4-b5e5-e401772d5543'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:16.014743 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7e198fb6-04b1-4ec4-b5e5-e401772d5543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed94a9a3ae8e4ceebb36fa398082fbe1', 'request_id': 'req-7e198fb6-04b1-4ec4-b5e5-e401772d5543'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:16.025649 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7e198fb6-04b1-4ec4-b5e5-e401772d5543 admin admin] The token cache is being invalidate because project ed94a9a3ae8e4ceebb36fa398082fbe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:50:16.025982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7e198fb6-04b1-4ec4-b5e5-e401772d5543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:16.026680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-7e198fb6-04b1-4ec4-b5e5-e401772d5543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:16.028240 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1389/2730] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:50:15 2022] DELETE /identity/v3/projects/ed94a9a3ae8e4ceebb36fa398082fbe1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:16.032377 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d2326-7e77-4b17-8a6f-8b6b0f3b0093 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:16.042807 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d2326-7e77-4b17-8a6f-8b6b0f3b0093 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:16.094498 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:16.101914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0658a-197a-40c9-ae8d-c6ef9f671bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:16.103309 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0658a-197a-40c9-ae8d-c6ef9f671bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:16.103683 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0658a-197a-40c9-ae8d-c6ef9f671bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:16.103971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0658a-197a-40c9-ae8d-c6ef9f671bc7 admin admin] PATH_INFO: `/v3/users/5eaaa2f595a74f84b57c36c346548cb8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:16.105706 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0658a-197a-40c9-ae8d-c6ef9f671bc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eaaa2f595a74f84b57c36c346548cb8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:16.108142 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0658a-197a-40c9-ae8d-c6ef9f671bc7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:16.155367 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2bc0658a-197a-40c9-ae8d-c6ef9f671bc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eaaa2f595a74f84b57c36c346548cb8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:16.167702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2bc0658a-197a-40c9-ae8d-c6ef9f671bc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eaaa2f595a74f84b57c36c346548cb8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:16.174526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2bc0658a-197a-40c9-ae8d-c6ef9f671bc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eaaa2f595a74f84b57c36c346548cb8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:16.175212 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2bc0658a-197a-40c9-ae8d-c6ef9f671bc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eaaa2f595a74f84b57c36c346548cb8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:16.184095 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2bc0658a-197a-40c9-ae8d-c6ef9f671bc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eaaa2f595a74f84b57c36c346548cb8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:16.190163 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1342/2731] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:50:16 2022] DELETE /identity/v3/users/5eaaa2f595a74f84b57c36c346548cb8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:16.462535 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e198fb6-04b1-4ec4-b5e5-e401772d5543 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:16.463992 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7e198fb6-04b1-4ec4-b5e5-e401772d5543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:16.477294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e198fb6-04b1-4ec4-b5e5-e401772d5543 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:16.536206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5834d6e-d7b8-404a-b943-6b5f5d32bf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:16.537073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5834d6e-d7b8-404a-b943-6b5f5d32bf66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:16.537390 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5834d6e-d7b8-404a-b943-6b5f5d32bf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:16.537672 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5834d6e-d7b8-404a-b943-6b5f5d32bf66 admin admin] PATH_INFO: `/v3/projects/720c61b0410f4a5e905efc6e29b8951f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:16.538904 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5834d6e-d7b8-404a-b943-6b5f5d32bf66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=720c61b0410f4a5e905efc6e29b8951f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:16.540876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5834d6e-d7b8-404a-b943-6b5f5d32bf66 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:16.564784 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:16.590014 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e5834d6e-d7b8-404a-b943-6b5f5d32bf66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '720c61b0410f4a5e905efc6e29b8951f', 'request_id': 'req-e5834d6e-d7b8-404a-b943-6b5f5d32bf66'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:16.597527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e5834d6e-d7b8-404a-b943-6b5f5d32bf66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '720c61b0410f4a5e905efc6e29b8951f', 'request_id': 'req-e5834d6e-d7b8-404a-b943-6b5f5d32bf66'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:16.603873 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e5834d6e-d7b8-404a-b943-6b5f5d32bf66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '720c61b0410f4a5e905efc6e29b8951f', 'request_id': 'req-e5834d6e-d7b8-404a-b943-6b5f5d32bf66'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:16.614276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e5834d6e-d7b8-404a-b943-6b5f5d32bf66 admin admin] The token cache is being invalidate because project 720c61b0410f4a5e905efc6e29b8951f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:50:16.614566 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e5834d6e-d7b8-404a-b943-6b5f5d32bf66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:16.615212 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e5834d6e-d7b8-404a-b943-6b5f5d32bf66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:16.616753 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1390/2732] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:50:16 2022] DELETE /identity/v3/projects/720c61b0410f4a5e905efc6e29b8951f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:16.623257 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79f5dc-ca05-42d5-9d0a-942b4ad2cdb0 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:16.623776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79f5dc-ca05-42d5-9d0a-942b4ad2cdb0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:16.623892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79f5dc-ca05-42d5-9d0a-942b4ad2cdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:16.642632 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4f79f5dc-ca05-42d5-9d0a-942b4ad2cdb0 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:16.643914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4f79f5dc-ca05-42d5-9d0a-942b4ad2cdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:16.706612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1343/2733] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:16.714448 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b19fcc-67b6-467e-b1a0-1905c20257ce None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:16.714739 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b19fcc-67b6-467e-b1a0-1905c20257ce None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:16.715019 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b19fcc-67b6-467e-b1a0-1905c20257ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:16.733720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-48b19fcc-67b6-467e-b1a0-1905c20257ce None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:16.734979 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-48b19fcc-67b6-467e-b1a0-1905c20257ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:16.744157 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1391/2734] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:16.749136 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79f5dc-ca05-42d5-9d0a-942b4ad2cdb0 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:16.762520 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79f5dc-ca05-42d5-9d0a-942b4ad2cdb0 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:16.768211 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012c591-9cfd-444b-a362-c069fd3316ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:16.768976 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012c591-9cfd-444b-a362-c069fd3316ff admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:16.769279 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012c591-9cfd-444b-a362-c069fd3316ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:16.769560 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012c591-9cfd-444b-a362-c069fd3316ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:16.770361 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012c591-9cfd-444b-a362-c069fd3316ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:50:16.770738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012c591-9cfd-444b-a362-c069fd3316ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:16.772425 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012c591-9cfd-444b-a362-c069fd3316ff admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:16.785179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1344/2735] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:50:16.790127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b19fcc-67b6-467e-b1a0-1905c20257ce None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:16.800332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b19fcc-67b6-467e-b1a0-1905c20257ce None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:16.807609 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e49ff9-17b5-493d-8cdf-560bf807ca44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:16.808270 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e49ff9-17b5-493d-8cdf-560bf807ca44 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:16.808580 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e49ff9-17b5-493d-8cdf-560bf807ca44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:16.808879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e49ff9-17b5-493d-8cdf-560bf807ca44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:16.809563 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e49ff9-17b5-493d-8cdf-560bf807ca44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:16.811521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e49ff9-17b5-493d-8cdf-560bf807ca44 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:16.838215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1392/2736] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:50:16 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:16.846150 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012c591-9cfd-444b-a362-c069fd3316ff admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:16.858131 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012c591-9cfd-444b-a362-c069fd3316ff admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:16.919273 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9abb9-f747-4b62-a473-17893719af14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:16.920021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9abb9-f747-4b62-a473-17893719af14 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:16.920294 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9abb9-f747-4b62-a473-17893719af14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:16.920576 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9abb9-f747-4b62-a473-17893719af14 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:16.921269 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9abb9-f747-4b62-a473-17893719af14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:16.923222 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9abb9-f747-4b62-a473-17893719af14 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:16.932379 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:16.946113 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1345/2737] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:50:16 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:50:16.950121 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e49ff9-17b5-493d-8cdf-560bf807ca44 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:16.960353 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e49ff9-17b5-493d-8cdf-560bf807ca44 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:16.964951 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03901b87-dda5-439b-b8c6-c407f0de90b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:16.965603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03901b87-dda5-439b-b8c6-c407f0de90b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:16.965883 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03901b87-dda5-439b-b8c6-c407f0de90b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:16.966191 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03901b87-dda5-439b-b8c6-c407f0de90b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:16.966961 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03901b87-dda5-439b-b8c6-c407f0de90b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:16.968585 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03901b87-dda5-439b-b8c6-c407f0de90b7 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:16.979536 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1393/2738] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:16.983747 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9abb9-f747-4b62-a473-17893719af14 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:16.993402 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9abb9-f747-4b62-a473-17893719af14 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:16.998723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43cebda-da99-4a1d-be57-10c630459810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:16.999387 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cebda-da99-4a1d-be57-10c630459810 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:16.999811 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cebda-da99-4a1d-be57-10c630459810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:16.999939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cebda-da99-4a1d-be57-10c630459810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:17.000706 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43cebda-da99-4a1d-be57-10c630459810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:17.002526 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43cebda-da99-4a1d-be57-10c630459810 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:17.013223 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1346/2739] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:17.017307 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03901b87-dda5-439b-b8c6-c407f0de90b7 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:17.027396 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03901b87-dda5-439b-b8c6-c407f0de90b7 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:17.032012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd0e84-e40f-4f35-8430-3058546daa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:17.032920 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd0e84-e40f-4f35-8430-3058546daa1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:17.033203 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd0e84-e40f-4f35-8430-3058546daa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:17.033480 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd0e84-e40f-4f35-8430-3058546daa1d admin admin] PATH_INFO: `/v3/projects/8156e0a1827642ef828e8b88343f088d/users/0c30e76da528404d964782e701cefce2/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:17.048226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd0e84-e40f-4f35-8430-3058546daa1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8156e0a1827642ef828e8b88343f088d, user_id=0c30e76da528404d964782e701cefce2, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:17.050975 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd0e84-e40f-4f35-8430-3058546daa1d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:17.064152 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1394/2740] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:50:17 2022] PUT /identity/v3/projects/8156e0a1827642ef828e8b88343f088d/users/0c30e76da528404d964782e701cefce2/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:17.074614 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab7967-81ff-4a14-8553-046497bfb982 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:17.074911 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab7967-81ff-4a14-8553-046497bfb982 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:17.075195 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab7967-81ff-4a14-8553-046497bfb982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:17.101806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-faab7967-81ff-4a14-8553-046497bfb982 None None] MFA Rules not processed for user `0c30e76da528404d964782e701cefce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:17.103572 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-faab7967-81ff-4a14-8553-046497bfb982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:17.167220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1347/2741] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:17.177849 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd0e84-e40f-4f35-8430-3058546daa1d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:17.179549 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8dfd0e84-e40f-4f35-8430-3058546daa1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:17.256810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd0e84-e40f-4f35-8430-3058546daa1d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:17.260488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18789efc-7fc7-4d7c-9345-f16aa2c5855e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:17.261133 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18789efc-7fc7-4d7c-9345-f16aa2c5855e service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:17.261417 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18789efc-7fc7-4d7c-9345-f16aa2c5855e service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:17.261693 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18789efc-7fc7-4d7c-9345-f16aa2c5855e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:17.270268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18789efc-7fc7-4d7c-9345-f16aa2c5855e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:17.271728 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18789efc-7fc7-4d7c-9345-f16aa2c5855e service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:17.283747 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1395/2742] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:19.676611 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e9368-527c-4ce4-8d02-36089746bad0 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:19.677151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e9368-527c-4ce4-8d02-36089746bad0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:19.677151 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e9368-527c-4ce4-8d02-36089746bad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:19.694725 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-387e9368-527c-4ce4-8d02-36089746bad0 None None] MFA Rules not processed for user `0c30e76da528404d964782e701cefce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:19.696041 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-387e9368-527c-4ce4-8d02-36089746bad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:19.703510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1348/2743] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:21.575956 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18789efc-7fc7-4d7c-9345-f16aa2c5855e service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:21.577482 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-18789efc-7fc7-4d7c-9345-f16aa2c5855e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:21.609665 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:21.645797 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18789efc-7fc7-4d7c-9345-f16aa2c5855e service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:21.651227 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d1cb4-38a1-416d-9a96-8ff3fa3652f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:21.652101 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d1cb4-38a1-416d-9a96-8ff3fa3652f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:21.652382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d1cb4-38a1-416d-9a96-8ff3fa3652f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:21.652649 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d1cb4-38a1-416d-9a96-8ff3fa3652f3 admin admin] PATH_INFO: `/v3/users/84c436bf88094a568a9a3e7ed761c4fa` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:21.653847 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d1cb4-38a1-416d-9a96-8ff3fa3652f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84c436bf88094a568a9a3e7ed761c4fa)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:21.655956 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d1cb4-38a1-416d-9a96-8ff3fa3652f3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:21.710103 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-836d1cb4-38a1-416d-9a96-8ff3fa3652f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84c436bf88094a568a9a3e7ed761c4fa'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:21.710766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-836d1cb4-38a1-416d-9a96-8ff3fa3652f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84c436bf88094a568a9a3e7ed761c4fa'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:21.711382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-836d1cb4-38a1-416d-9a96-8ff3fa3652f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84c436bf88094a568a9a3e7ed761c4fa'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:21.723987 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-836d1cb4-38a1-416d-9a96-8ff3fa3652f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84c436bf88094a568a9a3e7ed761c4fa'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:21.731389 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-836d1cb4-38a1-416d-9a96-8ff3fa3652f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84c436bf88094a568a9a3e7ed761c4fa'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:21.744222 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1396/2744] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:50:21 2022] DELETE /identity/v3/users/84c436bf88094a568a9a3e7ed761c4fa => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:21.748897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbc373-5688-4942-9022-073f916c5bf1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:21.749389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbc373-5688-4942-9022-073f916c5bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:21.749626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbc373-5688-4942-9022-073f916c5bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:21.776587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3afbc373-5688-4942-9022-073f916c5bf1 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:21.777874 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3afbc373-5688-4942-9022-073f916c5bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:21.838841 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1349/2745] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:50:21.848845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d1cb4-38a1-416d-9a96-8ff3fa3652f3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:21.850179 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-836d1cb4-38a1-416d-9a96-8ff3fa3652f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:21.912071 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d1cb4-38a1-416d-9a96-8ff3fa3652f3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:21.916401 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ac773-1a04-4b31-b73d-181e27186413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:21.917046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ac773-1a04-4b31-b73d-181e27186413 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:21.917323 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ac773-1a04-4b31-b73d-181e27186413 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:21.917618 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ac773-1a04-4b31-b73d-181e27186413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:21.926998 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ac773-1a04-4b31-b73d-181e27186413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:21.928567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ac773-1a04-4b31-b73d-181e27186413 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:21.939233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1397/2746] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:22.233371 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afbc373-5688-4942-9022-073f916c5bf1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:22.234863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3afbc373-5688-4942-9022-073f916c5bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:22.247261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afbc373-5688-4942-9022-073f916c5bf1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:22.253276 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be23a3-6343-4f92-9bc0-149f1c0fdfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:22.254187 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be23a3-6343-4f92-9bc0-149f1c0fdfcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:22.254476 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be23a3-6343-4f92-9bc0-149f1c0fdfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:22.254755 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be23a3-6343-4f92-9bc0-149f1c0fdfcb admin admin] PATH_INFO: `/v3/projects/6b42ffcbb47a464d8760cff525755b4e` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:22.255997 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be23a3-6343-4f92-9bc0-149f1c0fdfcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b42ffcbb47a464d8760cff525755b4e)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:22.258012 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be23a3-6343-4f92-9bc0-149f1c0fdfcb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:22.325208 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-71be23a3-6343-4f92-9bc0-149f1c0fdfcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b42ffcbb47a464d8760cff525755b4e', 'request_id': 'req-71be23a3-6343-4f92-9bc0-149f1c0fdfcb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:22.339121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-71be23a3-6343-4f92-9bc0-149f1c0fdfcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b42ffcbb47a464d8760cff525755b4e', 'request_id': 'req-71be23a3-6343-4f92-9bc0-149f1c0fdfcb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:22.340842 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:INSERT=2,keystone:SELECT=92,keystone:DELETE=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:22.348629 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-71be23a3-6343-4f92-9bc0-149f1c0fdfcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b42ffcbb47a464d8760cff525755b4e', 'request_id': 'req-71be23a3-6343-4f92-9bc0-149f1c0fdfcb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:22.361902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-71be23a3-6343-4f92-9bc0-149f1c0fdfcb admin admin] The token cache is being invalidate because project 6b42ffcbb47a464d8760cff525755b4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:50:22.362393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-71be23a3-6343-4f92-9bc0-149f1c0fdfcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:22.363148 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-71be23a3-6343-4f92-9bc0-149f1c0fdfcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:22.365017 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1350/2747] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:50:22 2022] DELETE /identity/v3/projects/6b42ffcbb47a464d8760cff525755b4e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:22.369876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd471e-b351-492a-9891-c7c88c073616 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:22.370427 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd471e-b351-492a-9891-c7c88c073616 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:22.373659 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd471e-b351-492a-9891-c7c88c073616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:22.393640 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-47fd471e-b351-492a-9891-c7c88c073616 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:22.394911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-47fd471e-b351-492a-9891-c7c88c073616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:22.455439 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:22.478122 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1398/2748] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:22.481245 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb639fa4-e4b5-4d24-8f35-adb3a730d2c3 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:22.481578 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb639fa4-e4b5-4d24-8f35-adb3a730d2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:22.481850 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb639fa4-e4b5-4d24-8f35-adb3a730d2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:22.518476 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-cb639fa4-e4b5-4d24-8f35-adb3a730d2c3 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:22.520130 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cb639fa4-e4b5-4d24-8f35-adb3a730d2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:22.529769 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1351/2749] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:22.534005 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd471e-b351-492a-9891-c7c88c073616 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:22.544547 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd471e-b351-492a-9891-c7c88c073616 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:22.549408 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac69f00-ed91-4138-81e9-6a9531b2ff2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:22.550206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac69f00-ed91-4138-81e9-6a9531b2ff2e admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:22.550531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac69f00-ed91-4138-81e9-6a9531b2ff2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:22.550956 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac69f00-ed91-4138-81e9-6a9531b2ff2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:22.551584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac69f00-ed91-4138-81e9-6a9531b2ff2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:50:22.551840 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac69f00-ed91-4138-81e9-6a9531b2ff2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:22.553494 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac69f00-ed91-4138-81e9-6a9531b2ff2e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:22.566574 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1399/2750] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:50:22.570789 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb639fa4-e4b5-4d24-8f35-adb3a730d2c3 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:22.581112 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb639fa4-e4b5-4d24-8f35-adb3a730d2c3 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:22.586009 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e595da-ad71-4e13-940e-46a9697079a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:22.586801 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e595da-ad71-4e13-940e-46a9697079a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:22.587122 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e595da-ad71-4e13-940e-46a9697079a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:22.587405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e595da-ad71-4e13-940e-46a9697079a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:22.588250 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e595da-ad71-4e13-940e-46a9697079a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:22.590249 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e595da-ad71-4e13-940e-46a9697079a1 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:22.629360 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1352/2751] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:50:22 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:22.630681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac69f00-ed91-4138-81e9-6a9531b2ff2e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:22.642426 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac69f00-ed91-4138-81e9-6a9531b2ff2e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:22.715930 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810b4ba-cfad-4abb-ae5f-91211327e53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:22.716696 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810b4ba-cfad-4abb-ae5f-91211327e53d admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:22.717035 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810b4ba-cfad-4abb-ae5f-91211327e53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:22.717368 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810b4ba-cfad-4abb-ae5f-91211327e53d admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:22.718141 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810b4ba-cfad-4abb-ae5f-91211327e53d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:22.720176 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810b4ba-cfad-4abb-ae5f-91211327e53d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:22.747319 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1400/2752] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:50:22 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:22.751152 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e595da-ad71-4e13-940e-46a9697079a1 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:22.766608 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e595da-ad71-4e13-940e-46a9697079a1 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:22.773457 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a38d60-09bd-4694-88d2-0d708e301883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:22.774243 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a38d60-09bd-4694-88d2-0d708e301883 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:22.774676 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a38d60-09bd-4694-88d2-0d708e301883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:22.775117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a38d60-09bd-4694-88d2-0d708e301883 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:22.775949 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a38d60-09bd-4694-88d2-0d708e301883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:22.777899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a38d60-09bd-4694-88d2-0d708e301883 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:22.797785 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1353/2753] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:22.802909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810b4ba-cfad-4abb-ae5f-91211327e53d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:22.821227 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810b4ba-cfad-4abb-ae5f-91211327e53d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:22.830499 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9b801-0f37-40f4-a707-1b27c333b292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:22.832354 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9b801-0f37-40f4-a707-1b27c333b292 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:22.832973 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9b801-0f37-40f4-a707-1b27c333b292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:22.833487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9b801-0f37-40f4-a707-1b27c333b292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:22.834777 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9b801-0f37-40f4-a707-1b27c333b292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:22.836829 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9b801-0f37-40f4-a707-1b27c333b292 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:22.849727 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1401/2754] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:22.854846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a38d60-09bd-4694-88d2-0d708e301883 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:22.873633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a38d60-09bd-4694-88d2-0d708e301883 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:22.879971 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d68bb-a3bc-44cc-9524-7cd3dbe76e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:22.881278 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d68bb-a3bc-44cc-9524-7cd3dbe76e7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:22.881815 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d68bb-a3bc-44cc-9524-7cd3dbe76e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:22.882539 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d68bb-a3bc-44cc-9524-7cd3dbe76e7f admin admin] PATH_INFO: `/v3/projects/d6b55b7006c340cca0c57f05cade9251/users/0fd41429ddfc4132a6b71d8b202a31ec/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:22.910793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d68bb-a3bc-44cc-9524-7cd3dbe76e7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6b55b7006c340cca0c57f05cade9251, user_id=0fd41429ddfc4132a6b71d8b202a31ec, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:22.913926 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d68bb-a3bc-44cc-9524-7cd3dbe76e7f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:22.950804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1354/2755] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:50:22 2022] PUT /identity/v3/projects/d6b55b7006c340cca0c57f05cade9251/users/0fd41429ddfc4132a6b71d8b202a31ec/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:22.951964 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc11b60-3026-44a6-8cb4-72db03a68245 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:22.952265 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc11b60-3026-44a6-8cb4-72db03a68245 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:22.952547 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc11b60-3026-44a6-8cb4-72db03a68245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:22.994935 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8fc11b60-3026-44a6-8cb4-72db03a68245 None None] MFA Rules not processed for user `0fd41429ddfc4132a6b71d8b202a31ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:22.996257 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8fc11b60-3026-44a6-8cb4-72db03a68245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:23.069484 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1402/2756] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:23.074148 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d68bb-a3bc-44cc-9524-7cd3dbe76e7f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:23.140611 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d68bb-a3bc-44cc-9524-7cd3dbe76e7f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:23.145326 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf97d42-72f0-4ab2-afd6-704e3d02fdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:23.146047 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf97d42-72f0-4ab2-afd6-704e3d02fdcb admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:23.146295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf97d42-72f0-4ab2-afd6-704e3d02fdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:23.146569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf97d42-72f0-4ab2-afd6-704e3d02fdcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:23.147369 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf97d42-72f0-4ab2-afd6-704e3d02fdcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:23.149281 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf97d42-72f0-4ab2-afd6-704e3d02fdcb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:23.178492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1355/2757] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:50:23 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:23.179735 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc11b60-3026-44a6-8cb4-72db03a68245 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:23.190637 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc11b60-3026-44a6-8cb4-72db03a68245 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:23.236046 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:23.250249 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65117ae-9e8c-4c71-9080-102adaf38c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:23.250923 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65117ae-9e8c-4c71-9080-102adaf38c59 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:23.251207 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65117ae-9e8c-4c71-9080-102adaf38c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:23.251483 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65117ae-9e8c-4c71-9080-102adaf38c59 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:23.252437 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65117ae-9e8c-4c71-9080-102adaf38c59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:23.254535 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65117ae-9e8c-4c71-9080-102adaf38c59 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:23.273675 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1403/2758] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:50:23 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:23.278233 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf97d42-72f0-4ab2-afd6-704e3d02fdcb admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:23.288441 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf97d42-72f0-4ab2-afd6-704e3d02fdcb admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:23.293708 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468be07-d3a0-4308-a27d-dc4225f22a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:23.294392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468be07-d3a0-4308-a27d-dc4225f22a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:23.294680 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468be07-d3a0-4308-a27d-dc4225f22a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:23.294956 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468be07-d3a0-4308-a27d-dc4225f22a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:23.295727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468be07-d3a0-4308-a27d-dc4225f22a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:23.297546 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468be07-d3a0-4308-a27d-dc4225f22a12 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:23.307986 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1356/2759] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:23.312009 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65117ae-9e8c-4c71-9080-102adaf38c59 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:23.322036 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65117ae-9e8c-4c71-9080-102adaf38c59 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:23.326599 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d2b28-a229-4eb8-abce-91e98afcfd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:23.327243 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d2b28-a229-4eb8-abce-91e98afcfd01 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:23.327527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d2b28-a229-4eb8-abce-91e98afcfd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:23.327799 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d2b28-a229-4eb8-abce-91e98afcfd01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:23.328513 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d2b28-a229-4eb8-abce-91e98afcfd01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:23.330128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d2b28-a229-4eb8-abce-91e98afcfd01 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:23.340233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1404/2760] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:23.344179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468be07-d3a0-4308-a27d-dc4225f22a12 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:23.354606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468be07-d3a0-4308-a27d-dc4225f22a12 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:23.359229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374509fd-a7c0-4012-83fa-dac084299547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:23.360095 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374509fd-a7c0-4012-83fa-dac084299547 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:23.360382 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374509fd-a7c0-4012-83fa-dac084299547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:23.360654 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374509fd-a7c0-4012-83fa-dac084299547 admin admin] PATH_INFO: `/v3/projects/319db005e29d41839a81e901e011e365/users/636a0ced37254aa1bd7bc486815184c6/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:23.374846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374509fd-a7c0-4012-83fa-dac084299547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=319db005e29d41839a81e901e011e365, user_id=636a0ced37254aa1bd7bc486815184c6, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:23.377723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374509fd-a7c0-4012-83fa-dac084299547 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:23.390578 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1357/2761] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:50:23 2022] PUT /identity/v3/projects/319db005e29d41839a81e901e011e365/users/636a0ced37254aa1bd7bc486815184c6/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:23.402648 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94524b0a-27bf-4749-8e46-b6db48617b0a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:23.402938 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94524b0a-27bf-4749-8e46-b6db48617b0a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:23.403212 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94524b0a-27bf-4749-8e46-b6db48617b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:23.429308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-94524b0a-27bf-4749-8e46-b6db48617b0a None None] MFA Rules not processed for user `636a0ced37254aa1bd7bc486815184c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:23.430720 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-94524b0a-27bf-4749-8e46-b6db48617b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:23.492425 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1405/2762] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:23.496894 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374509fd-a7c0-4012-83fa-dac084299547 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:23.531829 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:23.560963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374509fd-a7c0-4012-83fa-dac084299547 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:23.565598 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad89f8-953a-425e-8d42-51247f886c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:23.566282 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad89f8-953a-425e-8d42-51247f886c51 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:23.566568 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad89f8-953a-425e-8d42-51247f886c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:23.566843 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad89f8-953a-425e-8d42-51247f886c51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:23.567535 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad89f8-953a-425e-8d42-51247f886c51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:23.569442 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad89f8-953a-425e-8d42-51247f886c51 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:23.596171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1358/2763] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:50:23 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:23.600691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94524b0a-27bf-4749-8e46-b6db48617b0a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:23.611419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94524b0a-27bf-4749-8e46-b6db48617b0a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:23.670018 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e8ca8-b70b-4bc4-a705-66a97872bb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:23.670818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e8ca8-b70b-4bc4-a705-66a97872bb72 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:23.671049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e8ca8-b70b-4bc4-a705-66a97872bb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:23.671495 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e8ca8-b70b-4bc4-a705-66a97872bb72 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:23.672195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e8ca8-b70b-4bc4-a705-66a97872bb72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:23.674311 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e8ca8-b70b-4bc4-a705-66a97872bb72 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:23.698365 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1406/2764] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:50:23 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:50:23.702474 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad89f8-953a-425e-8d42-51247f886c51 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:23.713391 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad89f8-953a-425e-8d42-51247f886c51 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:23.717914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f1eef-2a25-4187-a819-f72f19f219f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:23.718606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f1eef-2a25-4187-a819-f72f19f219f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:23.718888 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f1eef-2a25-4187-a819-f72f19f219f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:23.719183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f1eef-2a25-4187-a819-f72f19f219f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:23.719900 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f1eef-2a25-4187-a819-f72f19f219f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:23.721474 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f1eef-2a25-4187-a819-f72f19f219f5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:23.731723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1359/2765] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:23.735699 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e8ca8-b70b-4bc4-a705-66a97872bb72 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:23.745583 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e8ca8-b70b-4bc4-a705-66a97872bb72 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:23.750294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfc078-8348-4834-94d6-802727dcd5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:23.751163 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfc078-8348-4834-94d6-802727dcd5f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:23.751455 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfc078-8348-4834-94d6-802727dcd5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:23.751734 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfc078-8348-4834-94d6-802727dcd5f1 admin admin] PATH_INFO: `/v3/projects/c7d97a84c9674ceda3de1386519e0a14/users/b681352dd54c4d17b5239bf8395d6126/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:23.766749 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecfc078-8348-4834-94d6-802727dcd5f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7d97a84c9674ceda3de1386519e0a14, user_id=b681352dd54c4d17b5239bf8395d6126, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:23.769384 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecfc078-8348-4834-94d6-802727dcd5f1 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:23.783872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1407/2766] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:50:23 2022] PUT /identity/v3/projects/c7d97a84c9674ceda3de1386519e0a14/users/b681352dd54c4d17b5239bf8395d6126/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:23.787878 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f1eef-2a25-4187-a819-f72f19f219f5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:23.797699 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f1eef-2a25-4187-a819-f72f19f219f5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:23.853393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b514057-04b3-4ec4-8a9a-0155f0d43f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:23.854616 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b514057-04b3-4ec4-8a9a-0155f0d43f3f admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:23.854902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b514057-04b3-4ec4-8a9a-0155f0d43f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:23.855178 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b514057-04b3-4ec4-8a9a-0155f0d43f3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:23.856037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b514057-04b3-4ec4-8a9a-0155f0d43f3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:23.857635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b514057-04b3-4ec4-8a9a-0155f0d43f3f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:23.867865 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1360/2767] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:23.872110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfc078-8348-4834-94d6-802727dcd5f1 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:23.882683 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfc078-8348-4834-94d6-802727dcd5f1 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:23.887741 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73655240-77b1-4381-bd1b-a3358bcf9c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:23.888611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73655240-77b1-4381-bd1b-a3358bcf9c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:23.888893 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73655240-77b1-4381-bd1b-a3358bcf9c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:23.889171 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73655240-77b1-4381-bd1b-a3358bcf9c86 admin admin] PATH_INFO: `/v3/projects/c7d97a84c9674ceda3de1386519e0a14/users/b681352dd54c4d17b5239bf8395d6126/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:23.891178 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73655240-77b1-4381-bd1b-a3358bcf9c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7d97a84c9674ceda3de1386519e0a14, user_id=b681352dd54c4d17b5239bf8395d6126, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:23.893791 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73655240-77b1-4381-bd1b-a3358bcf9c86 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:23.904472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1408/2768] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:50:23 2022] PUT /identity/v3/projects/c7d97a84c9674ceda3de1386519e0a14/users/b681352dd54c4d17b5239bf8395d6126/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:23.914593 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328dde1-9721-48c1-bdac-696afc09845e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:23.914884 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328dde1-9721-48c1-bdac-696afc09845e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:23.915179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328dde1-9721-48c1-bdac-696afc09845e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:23.940960 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-6328dde1-9721-48c1-bdac-696afc09845e None None] MFA Rules not processed for user `b681352dd54c4d17b5239bf8395d6126`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:23.942203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6328dde1-9721-48c1-bdac-696afc09845e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:24.011331 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1361/2769] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:50:24.021522 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73655240-77b1-4381-bd1b-a3358bcf9c86 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:24.023268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-73655240-77b1-4381-bd1b-a3358bcf9c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:24.033158 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:24.085310 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73655240-77b1-4381-bd1b-a3358bcf9c86 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:24.088796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef8d73-ddc8-4b37-ad69-a58be8bbf193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:24.089452 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef8d73-ddc8-4b37-ad69-a58be8bbf193 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:24.089737 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef8d73-ddc8-4b37-ad69-a58be8bbf193 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:24.090049 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef8d73-ddc8-4b37-ad69-a58be8bbf193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:24.099499 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef8d73-ddc8-4b37-ad69-a58be8bbf193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:24.100957 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef8d73-ddc8-4b37-ad69-a58be8bbf193 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:24.111774 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1409/2770] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:24.156252 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328dde1-9721-48c1-bdac-696afc09845e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:24.157696 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6328dde1-9721-48c1-bdac-696afc09845e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:24.171332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328dde1-9721-48c1-bdac-696afc09845e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:24.175208 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb3f46-cf64-4e34-9058-dc015d3a8ac0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:24.175948 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb3f46-cf64-4e34-9058-dc015d3a8ac0 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:24.176242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb3f46-cf64-4e34-9058-dc015d3a8ac0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:24.176522 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb3f46-cf64-4e34-9058-dc015d3a8ac0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:24.209660 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:24.246731 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb3f46-cf64-4e34-9058-dc015d3a8ac0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:24.248409 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb3f46-cf64-4e34-9058-dc015d3a8ac0 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:24.260811 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1362/2771] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:27.152127 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef8d73-ddc8-4b37-ad69-a58be8bbf193 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:27.214902 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef8d73-ddc8-4b37-ad69-a58be8bbf193 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:27.219450 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3792bf9-c956-4597-ba26-760d1f0e7db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:27.220313 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3792bf9-c956-4597-ba26-760d1f0e7db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:27.220598 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3792bf9-c956-4597-ba26-760d1f0e7db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:27.220885 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3792bf9-c956-4597-ba26-760d1f0e7db5 admin admin] PATH_INFO: `/v3/users/0fd41429ddfc4132a6b71d8b202a31ec` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:27.222095 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3792bf9-c956-4597-ba26-760d1f0e7db5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fd41429ddfc4132a6b71d8b202a31ec)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:27.224013 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3792bf9-c956-4597-ba26-760d1f0e7db5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:27.268574 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e3792bf9-c956-4597-ba26-760d1f0e7db5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fd41429ddfc4132a6b71d8b202a31ec'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:27.269225 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e3792bf9-c956-4597-ba26-760d1f0e7db5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fd41429ddfc4132a6b71d8b202a31ec'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:27.269829 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e3792bf9-c956-4597-ba26-760d1f0e7db5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fd41429ddfc4132a6b71d8b202a31ec'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:27.282713 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e3792bf9-c956-4597-ba26-760d1f0e7db5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fd41429ddfc4132a6b71d8b202a31ec'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:27.289921 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e3792bf9-c956-4597-ba26-760d1f0e7db5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fd41429ddfc4132a6b71d8b202a31ec'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:27.304024 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1410/2772] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:50:27 2022] DELETE /identity/v3/users/0fd41429ddfc4132a6b71d8b202a31ec => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:27.308454 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dcd1a-45d2-4410-ac27-9896e4688289 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:27.308953 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dcd1a-45d2-4410-ac27-9896e4688289 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:27.309066 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dcd1a-45d2-4410-ac27-9896e4688289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:27.326362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-400dcd1a-45d2-4410-ac27-9896e4688289 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:27.327485 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-400dcd1a-45d2-4410-ac27-9896e4688289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:27.388278 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1363/2773] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:27.398037 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3792bf9-c956-4597-ba26-760d1f0e7db5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:27.399453 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e3792bf9-c956-4597-ba26-760d1f0e7db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:27.441588 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:27.469329 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3792bf9-c956-4597-ba26-760d1f0e7db5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:27.472980 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0156556-059e-4a0e-a49f-b1e5c994bbb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:27.473620 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0156556-059e-4a0e-a49f-b1e5c994bbb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:27.473906 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0156556-059e-4a0e-a49f-b1e5c994bbb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:27.474217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0156556-059e-4a0e-a49f-b1e5c994bbb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:27.483176 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0156556-059e-4a0e-a49f-b1e5c994bbb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:27.484613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0156556-059e-4a0e-a49f-b1e5c994bbb2 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:27.494293 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1411/2774] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:27.767317 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dcd1a-45d2-4410-ac27-9896e4688289 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:27.768970 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-400dcd1a-45d2-4410-ac27-9896e4688289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:27.788928 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dcd1a-45d2-4410-ac27-9896e4688289 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:27.794091 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03490112-46f8-4cf8-af44-f7d567ba7ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:27.794968 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03490112-46f8-4cf8-af44-f7d567ba7ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:27.795252 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03490112-46f8-4cf8-af44-f7d567ba7ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:27.795531 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03490112-46f8-4cf8-af44-f7d567ba7ce3 admin admin] PATH_INFO: `/v3/projects/d6b55b7006c340cca0c57f05cade9251` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:27.796739 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03490112-46f8-4cf8-af44-f7d567ba7ce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6b55b7006c340cca0c57f05cade9251)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:27.798763 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03490112-46f8-4cf8-af44-f7d567ba7ce3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:27.859914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-03490112-46f8-4cf8-af44-f7d567ba7ce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6b55b7006c340cca0c57f05cade9251', 'request_id': 'req-03490112-46f8-4cf8-af44-f7d567ba7ce3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:27.871493 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-03490112-46f8-4cf8-af44-f7d567ba7ce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6b55b7006c340cca0c57f05cade9251', 'request_id': 'req-03490112-46f8-4cf8-af44-f7d567ba7ce3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:27.881920 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-03490112-46f8-4cf8-af44-f7d567ba7ce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6b55b7006c340cca0c57f05cade9251', 'request_id': 'req-03490112-46f8-4cf8-af44-f7d567ba7ce3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:27.894976 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-03490112-46f8-4cf8-af44-f7d567ba7ce3 admin admin] The token cache is being invalidate because project d6b55b7006c340cca0c57f05cade9251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:50:27.895380 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-03490112-46f8-4cf8-af44-f7d567ba7ce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:27.895922 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-03490112-46f8-4cf8-af44-f7d567ba7ce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:27.897682 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1364/2775] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:50:27 2022] DELETE /identity/v3/projects/d6b55b7006c340cca0c57f05cade9251 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:27.901639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0156556-059e-4a0e-a49f-b1e5c994bbb2 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:27.912642 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0156556-059e-4a0e-a49f-b1e5c994bbb2 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:27.976169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1703ede-73c3-40e6-a6e0-1620ca226b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:27.977040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1703ede-73c3-40e6-a6e0-1620ca226b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:27.977324 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1703ede-73c3-40e6-a6e0-1620ca226b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:27.977625 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1703ede-73c3-40e6-a6e0-1620ca226b01 admin admin] PATH_INFO: `/v3/users/636a0ced37254aa1bd7bc486815184c6` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:27.978850 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1703ede-73c3-40e6-a6e0-1620ca226b01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=636a0ced37254aa1bd7bc486815184c6)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:27.980768 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1703ede-73c3-40e6-a6e0-1620ca226b01 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:28.028005 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b1703ede-73c3-40e6-a6e0-1620ca226b01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '636a0ced37254aa1bd7bc486815184c6'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:28.028661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b1703ede-73c3-40e6-a6e0-1620ca226b01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '636a0ced37254aa1bd7bc486815184c6'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:28.029272 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b1703ede-73c3-40e6-a6e0-1620ca226b01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '636a0ced37254aa1bd7bc486815184c6'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:28.041902 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b1703ede-73c3-40e6-a6e0-1620ca226b01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '636a0ced37254aa1bd7bc486815184c6'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:28.049322 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b1703ede-73c3-40e6-a6e0-1620ca226b01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '636a0ced37254aa1bd7bc486815184c6'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:28.062277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1412/2776] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:50:27 2022] DELETE /identity/v3/users/636a0ced37254aa1bd7bc486815184c6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:28.338343 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03490112-46f8-4cf8-af44-f7d567ba7ce3 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:28.339817 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-03490112-46f8-4cf8-af44-f7d567ba7ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:28.355216 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03490112-46f8-4cf8-af44-f7d567ba7ce3 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:28.403470 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:28.415704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5490c-b421-43cf-83cb-f4726d3aff62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:28.416614 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5490c-b421-43cf-83cb-f4726d3aff62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:28.416903 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5490c-b421-43cf-83cb-f4726d3aff62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:28.417179 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5490c-b421-43cf-83cb-f4726d3aff62 admin admin] PATH_INFO: `/v3/projects/319db005e29d41839a81e901e011e365` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:28.418393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5490c-b421-43cf-83cb-f4726d3aff62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=319db005e29d41839a81e901e011e365)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:28.420334 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5490c-b421-43cf-83cb-f4726d3aff62 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:28.468658 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-fcf5490c-b421-43cf-83cb-f4726d3aff62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '319db005e29d41839a81e901e011e365', 'request_id': 'req-fcf5490c-b421-43cf-83cb-f4726d3aff62'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:28.474606 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-fcf5490c-b421-43cf-83cb-f4726d3aff62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '319db005e29d41839a81e901e011e365', 'request_id': 'req-fcf5490c-b421-43cf-83cb-f4726d3aff62'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:28.482820 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-fcf5490c-b421-43cf-83cb-f4726d3aff62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '319db005e29d41839a81e901e011e365', 'request_id': 'req-fcf5490c-b421-43cf-83cb-f4726d3aff62'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:28.494566 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-fcf5490c-b421-43cf-83cb-f4726d3aff62 admin admin] The token cache is being invalidate because project 319db005e29d41839a81e901e011e365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:50:28.495042 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-fcf5490c-b421-43cf-83cb-f4726d3aff62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:28.495785 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-fcf5490c-b421-43cf-83cb-f4726d3aff62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:28.497436 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1365/2777] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:50:28 2022] DELETE /identity/v3/projects/319db005e29d41839a81e901e011e365 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:28.501366 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1703ede-73c3-40e6-a6e0-1620ca226b01 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:28.503440 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b1703ede-73c3-40e6-a6e0-1620ca226b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:28.516435 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1703ede-73c3-40e6-a6e0-1620ca226b01 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:28.564862 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:28.576282 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db4127-bc6f-4c3f-8486-cb756e5d15d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:28.577157 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db4127-bc6f-4c3f-8486-cb756e5d15d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:28.577438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db4127-bc6f-4c3f-8486-cb756e5d15d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:28.577713 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db4127-bc6f-4c3f-8486-cb756e5d15d2 admin admin] PATH_INFO: `/v3/users/b681352dd54c4d17b5239bf8395d6126` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:28.578925 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db4127-bc6f-4c3f-8486-cb756e5d15d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b681352dd54c4d17b5239bf8395d6126)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:28.580891 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db4127-bc6f-4c3f-8486-cb756e5d15d2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:28.627424 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-06db4127-bc6f-4c3f-8486-cb756e5d15d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b681352dd54c4d17b5239bf8395d6126'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:28.628007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-06db4127-bc6f-4c3f-8486-cb756e5d15d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b681352dd54c4d17b5239bf8395d6126'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:28.628611 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-06db4127-bc6f-4c3f-8486-cb756e5d15d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b681352dd54c4d17b5239bf8395d6126'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:28.640434 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-06db4127-bc6f-4c3f-8486-cb756e5d15d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b681352dd54c4d17b5239bf8395d6126'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:28.647138 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-06db4127-bc6f-4c3f-8486-cb756e5d15d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b681352dd54c4d17b5239bf8395d6126'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:28.659931 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1413/2778] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:50:28 2022] DELETE /identity/v3/users/b681352dd54c4d17b5239bf8395d6126 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:28.930244 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5490c-b421-43cf-83cb-f4726d3aff62 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:28.932380 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fcf5490c-b421-43cf-83cb-f4726d3aff62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:28.945596 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5490c-b421-43cf-83cb-f4726d3aff62 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:29.004799 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380edf0-84fe-4614-b983-e3dff25fca65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:29.005668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380edf0-84fe-4614-b983-e3dff25fca65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:29.005947 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380edf0-84fe-4614-b983-e3dff25fca65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:29.006251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380edf0-84fe-4614-b983-e3dff25fca65 admin admin] PATH_INFO: `/v3/projects/c7d97a84c9674ceda3de1386519e0a14` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:29.007469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380edf0-84fe-4614-b983-e3dff25fca65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7d97a84c9674ceda3de1386519e0a14)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:29.009420 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380edf0-84fe-4614-b983-e3dff25fca65 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:29.055634 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7380edf0-84fe-4614-b983-e3dff25fca65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7d97a84c9674ceda3de1386519e0a14', 'request_id': 'req-7380edf0-84fe-4614-b983-e3dff25fca65'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:29.062610 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7380edf0-84fe-4614-b983-e3dff25fca65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7d97a84c9674ceda3de1386519e0a14', 'request_id': 'req-7380edf0-84fe-4614-b983-e3dff25fca65'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:29.070435 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7380edf0-84fe-4614-b983-e3dff25fca65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7d97a84c9674ceda3de1386519e0a14', 'request_id': 'req-7380edf0-84fe-4614-b983-e3dff25fca65'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:29.081668 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7380edf0-84fe-4614-b983-e3dff25fca65 admin admin] The token cache is being invalidate because project c7d97a84c9674ceda3de1386519e0a14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:50:29.081974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7380edf0-84fe-4614-b983-e3dff25fca65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:29.082616 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7380edf0-84fe-4614-b983-e3dff25fca65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:29.084145 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1366/2779] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:50:28 2022] DELETE /identity/v3/projects/c7d97a84c9674ceda3de1386519e0a14 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:29.089985 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839ccb2-dc76-4f94-bb06-246b9bd72abf None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:29.090281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839ccb2-dc76-4f94-bb06-246b9bd72abf None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:29.090558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839ccb2-dc76-4f94-bb06-246b9bd72abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:29.110414 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-e839ccb2-dc76-4f94-bb06-246b9bd72abf None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:29.112102 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e839ccb2-dc76-4f94-bb06-246b9bd72abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:29.175948 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1414/2780] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:29.184021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363773e9-5c9e-4b79-af2d-f19046d2207a None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:29.184327 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363773e9-5c9e-4b79-af2d-f19046d2207a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:29.184939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363773e9-5c9e-4b79-af2d-f19046d2207a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:29.202687 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-363773e9-5c9e-4b79-af2d-f19046d2207a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:29.203878 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-363773e9-5c9e-4b79-af2d-f19046d2207a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:29.212600 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1367/2781] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:29.216596 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839ccb2-dc76-4f94-bb06-246b9bd72abf None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:29.226225 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839ccb2-dc76-4f94-bb06-246b9bd72abf None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:29.231632 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31cc11-06c1-4974-96bd-3570927c6836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:29.232281 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31cc11-06c1-4974-96bd-3570927c6836 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:29.232570 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31cc11-06c1-4974-96bd-3570927c6836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:29.232853 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31cc11-06c1-4974-96bd-3570927c6836 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:29.233581 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31cc11-06c1-4974-96bd-3570927c6836 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:50:29.233893 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31cc11-06c1-4974-96bd-3570927c6836 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:29.235528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31cc11-06c1-4974-96bd-3570927c6836 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:29.246817 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1415/2782] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:50:29.251056 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363773e9-5c9e-4b79-af2d-f19046d2207a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:29.256867 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:29.260880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363773e9-5c9e-4b79-af2d-f19046d2207a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:29.266969 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4071a-44df-4387-a96a-2af6baaa4298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:29.267784 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4071a-44df-4387-a96a-2af6baaa4298 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:29.268231 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4071a-44df-4387-a96a-2af6baaa4298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:29.268568 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4071a-44df-4387-a96a-2af6baaa4298 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:29.269468 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4071a-44df-4387-a96a-2af6baaa4298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:29.271467 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4071a-44df-4387-a96a-2af6baaa4298 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:29.297354 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1368/2783] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:50:29 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:50:29.301471 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31cc11-06c1-4974-96bd-3570927c6836 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:29.311647 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31cc11-06c1-4974-96bd-3570927c6836 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:29.361737 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:29.368268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34fda9-93f8-4255-b475-5384a49d2c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:29.368946 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34fda9-93f8-4255-b475-5384a49d2c73 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:29.369530 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34fda9-93f8-4255-b475-5384a49d2c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:29.369815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34fda9-93f8-4255-b475-5384a49d2c73 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:29.370597 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34fda9-93f8-4255-b475-5384a49d2c73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:29.372832 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34fda9-93f8-4255-b475-5384a49d2c73 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:29.393039 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1416/2784] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:50:29 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:29.397076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4071a-44df-4387-a96a-2af6baaa4298 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:29.408076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4071a-44df-4387-a96a-2af6baaa4298 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:29.412673 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e1048-6623-429c-9c77-be7ed55a5f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:29.413323 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e1048-6623-429c-9c77-be7ed55a5f95 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:29.413599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e1048-6623-429c-9c77-be7ed55a5f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:29.413871 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e1048-6623-429c-9c77-be7ed55a5f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:29.414714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e1048-6623-429c-9c77-be7ed55a5f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:29.416910 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e1048-6623-429c-9c77-be7ed55a5f95 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:29.427106 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1369/2785] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:29.430994 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34fda9-93f8-4255-b475-5384a49d2c73 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:29.440860 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34fda9-93f8-4255-b475-5384a49d2c73 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:29.445808 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98814a-4dde-4913-9d60-77fc4f7563e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:29.446485 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98814a-4dde-4913-9d60-77fc4f7563e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:29.446772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98814a-4dde-4913-9d60-77fc4f7563e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:29.447050 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98814a-4dde-4913-9d60-77fc4f7563e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:29.447770 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98814a-4dde-4913-9d60-77fc4f7563e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:29.449377 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98814a-4dde-4913-9d60-77fc4f7563e1 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:29.459332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1417/2786] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:29.463722 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e1048-6623-429c-9c77-be7ed55a5f95 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:29.473405 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e1048-6623-429c-9c77-be7ed55a5f95 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:29.477924 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf02f10-5c38-4ecf-bd1f-3a912718ac69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:29.478819 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf02f10-5c38-4ecf-bd1f-3a912718ac69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:29.479673 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf02f10-5c38-4ecf-bd1f-3a912718ac69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:29.479972 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf02f10-5c38-4ecf-bd1f-3a912718ac69 admin admin] PATH_INFO: `/v3/projects/9df9bfc93ea34c3892ef94ceeebbf54b/users/bfd1c853ef064ae28fcbe8d3435c48e8/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:29.495065 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf02f10-5c38-4ecf-bd1f-3a912718ac69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df9bfc93ea34c3892ef94ceeebbf54b, user_id=bfd1c853ef064ae28fcbe8d3435c48e8, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:29.497692 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf02f10-5c38-4ecf-bd1f-3a912718ac69 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:29.511074 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1370/2787] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:50:29 2022] PUT /identity/v3/projects/9df9bfc93ea34c3892ef94ceeebbf54b/users/bfd1c853ef064ae28fcbe8d3435c48e8/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:50:29.523218 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588feb1c-65dd-4ed2-af4a-89c66ecf9559 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:29.523708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588feb1c-65dd-4ed2-af4a-89c66ecf9559 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:29.523824 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588feb1c-65dd-4ed2-af4a-89c66ecf9559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:29.550115 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-588feb1c-65dd-4ed2-af4a-89c66ecf9559 None None] MFA Rules not processed for user `bfd1c853ef064ae28fcbe8d3435c48e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:29.551333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-588feb1c-65dd-4ed2-af4a-89c66ecf9559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:29.614259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1418/2788] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:29.618456 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf02f10-5c38-4ecf-bd1f-3a912718ac69 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:29.689574 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf02f10-5c38-4ecf-bd1f-3a912718ac69 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:29.695140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a518ddc-6ba5-4bbe-8a23-69ca1d4922cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:29.695876 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a518ddc-6ba5-4bbe-8a23-69ca1d4922cd admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:29.696974 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a518ddc-6ba5-4bbe-8a23-69ca1d4922cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:29.697102 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a518ddc-6ba5-4bbe-8a23-69ca1d4922cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:29.697841 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a518ddc-6ba5-4bbe-8a23-69ca1d4922cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:29.699848 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a518ddc-6ba5-4bbe-8a23-69ca1d4922cd admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:29.730585 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1371/2789] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:50:29 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:29.734541 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588feb1c-65dd-4ed2-af4a-89c66ecf9559 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:29.745857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588feb1c-65dd-4ed2-af4a-89c66ecf9559 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:29.809016 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c88439-7d82-45d3-8467-2fee7456eef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:29.809782 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c88439-7d82-45d3-8467-2fee7456eef7 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:29.810173 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c88439-7d82-45d3-8467-2fee7456eef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:29.810515 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c88439-7d82-45d3-8467-2fee7456eef7 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:29.811276 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c88439-7d82-45d3-8467-2fee7456eef7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:29.813406 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c88439-7d82-45d3-8467-2fee7456eef7 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:29.833881 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1419/2790] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:50:29 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:29.837863 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a518ddc-6ba5-4bbe-8a23-69ca1d4922cd admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:29.853633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a518ddc-6ba5-4bbe-8a23-69ca1d4922cd admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:29.859655 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1d1bf-d29c-4fa2-818a-f10f0211a6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:29.860803 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1d1bf-d29c-4fa2-818a-f10f0211a6e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:29.860803 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1d1bf-d29c-4fa2-818a-f10f0211a6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:29.861139 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1d1bf-d29c-4fa2-818a-f10f0211a6e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:29.861798 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f1d1bf-d29c-4fa2-818a-f10f0211a6e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:29.865018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f1d1bf-d29c-4fa2-818a-f10f0211a6e4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:29.882022 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1372/2791] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:29.883470 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c88439-7d82-45d3-8467-2fee7456eef7 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:29.897018 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c88439-7d82-45d3-8467-2fee7456eef7 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:29.902544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c93d9-a2d7-4ea0-97f1-13d6e04efe21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:29.903302 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c93d9-a2d7-4ea0-97f1-13d6e04efe21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:29.903584 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c93d9-a2d7-4ea0-97f1-13d6e04efe21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:29.903855 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c93d9-a2d7-4ea0-97f1-13d6e04efe21 admin admin] PATH_INFO: `/v3/projects/ee4657cc40204884baaeb3a30f11d322/users/11d8b2dfba394b91b79d3b6a74bcb8ad/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:29.919855 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c93d9-a2d7-4ea0-97f1-13d6e04efe21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee4657cc40204884baaeb3a30f11d322, user_id=11d8b2dfba394b91b79d3b6a74bcb8ad, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:29.922673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c93d9-a2d7-4ea0-97f1-13d6e04efe21 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:29.938017 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1420/2792] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:50:29 2022] PUT /identity/v3/projects/ee4657cc40204884baaeb3a30f11d322/users/11d8b2dfba394b91b79d3b6a74bcb8ad/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:29.942089 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1d1bf-d29c-4fa2-818a-f10f0211a6e4 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:29.952648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1d1bf-d29c-4fa2-818a-f10f0211a6e4 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:30.008469 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e9034-8abb-4e1e-ad7c-360289ef5dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:30.009152 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e9034-8abb-4e1e-ad7c-360289ef5dfb admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:30.009435 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e9034-8abb-4e1e-ad7c-360289ef5dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:30.009716 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e9034-8abb-4e1e-ad7c-360289ef5dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:30.010461 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e9034-8abb-4e1e-ad7c-360289ef5dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:30.012048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e9034-8abb-4e1e-ad7c-360289ef5dfb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:30.021578 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1373/2793] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:30.026217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c93d9-a2d7-4ea0-97f1-13d6e04efe21 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:30.036286 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c93d9-a2d7-4ea0-97f1-13d6e04efe21 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:30.041498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b1add-8863-4f06-856d-0ab80e572d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:30.042398 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b1add-8863-4f06-856d-0ab80e572d45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:30.042679 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b1add-8863-4f06-856d-0ab80e572d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:30.042950 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b1add-8863-4f06-856d-0ab80e572d45 admin admin] PATH_INFO: `/v3/projects/ee4657cc40204884baaeb3a30f11d322/users/11d8b2dfba394b91b79d3b6a74bcb8ad/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:30.044901 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b1add-8863-4f06-856d-0ab80e572d45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee4657cc40204884baaeb3a30f11d322, user_id=11d8b2dfba394b91b79d3b6a74bcb8ad, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:30.047567 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b1add-8863-4f06-856d-0ab80e572d45 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:30.058442 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1421/2794] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:50:30 2022] PUT /identity/v3/projects/ee4657cc40204884baaeb3a30f11d322/users/11d8b2dfba394b91b79d3b6a74bcb8ad/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:30.068119 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8498ab-a44d-4a2a-bb5a-30fcd53186a6 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:30.068434 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8498ab-a44d-4a2a-bb5a-30fcd53186a6 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:30.068698 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8498ab-a44d-4a2a-bb5a-30fcd53186a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:30.094708 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-de8498ab-a44d-4a2a-bb5a-30fcd53186a6 None None] MFA Rules not processed for user `11d8b2dfba394b91b79d3b6a74bcb8ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:30.095944 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-de8498ab-a44d-4a2a-bb5a-30fcd53186a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:30.100872 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:30.163702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1374/2795] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:30.174768 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b1add-8863-4f06-856d-0ab80e572d45 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:30.176527 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-935b1add-8863-4f06-856d-0ab80e572d45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:30.198978 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:30.237345 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b1add-8863-4f06-856d-0ab80e572d45 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:30.240967 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b4784e-926d-49c2-8e1b-61aee5e6137a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:30.241662 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4784e-926d-49c2-8e1b-61aee5e6137a service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:30.241946 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4784e-926d-49c2-8e1b-61aee5e6137a service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:30.242272 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4784e-926d-49c2-8e1b-61aee5e6137a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:30.252600 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b4784e-926d-49c2-8e1b-61aee5e6137a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:30.254188 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b4784e-926d-49c2-8e1b-61aee5e6137a service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:30.265092 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1422/2796] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:30.646662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8498ab-a44d-4a2a-bb5a-30fcd53186a6 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:30.711470 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8498ab-a44d-4a2a-bb5a-30fcd53186a6 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:30.716717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459a8d0-ca9d-4ddf-8044-3949c8827beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:30.717593 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459a8d0-ca9d-4ddf-8044-3949c8827beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:30.717880 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459a8d0-ca9d-4ddf-8044-3949c8827beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:30.718210 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459a8d0-ca9d-4ddf-8044-3949c8827beb admin admin] PATH_INFO: `/v3/users/bfd1c853ef064ae28fcbe8d3435c48e8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:30.719587 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459a8d0-ca9d-4ddf-8044-3949c8827beb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfd1c853ef064ae28fcbe8d3435c48e8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:30.721525 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459a8d0-ca9d-4ddf-8044-3949c8827beb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:30.768751 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6459a8d0-ca9d-4ddf-8044-3949c8827beb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfd1c853ef064ae28fcbe8d3435c48e8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:30.781037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6459a8d0-ca9d-4ddf-8044-3949c8827beb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfd1c853ef064ae28fcbe8d3435c48e8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:30.787731 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6459a8d0-ca9d-4ddf-8044-3949c8827beb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfd1c853ef064ae28fcbe8d3435c48e8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:30.788392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6459a8d0-ca9d-4ddf-8044-3949c8827beb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfd1c853ef064ae28fcbe8d3435c48e8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:30.797495 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6459a8d0-ca9d-4ddf-8044-3949c8827beb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfd1c853ef064ae28fcbe8d3435c48e8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:30.804104 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1375/2797] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:50:30 2022] DELETE /identity/v3/users/bfd1c853ef064ae28fcbe8d3435c48e8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:30.809656 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0972d-88d3-4925-90ae-ed921efc61b4 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:30.810155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0972d-88d3-4925-90ae-ed921efc61b4 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:30.810655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0972d-88d3-4925-90ae-ed921efc61b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:30.829524 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-79f0972d-88d3-4925-90ae-ed921efc61b4 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:30.830865 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-79f0972d-88d3-4925-90ae-ed921efc61b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:30.891022 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1423/2798] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:30.901136 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459a8d0-ca9d-4ddf-8044-3949c8827beb admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:30.902552 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6459a8d0-ca9d-4ddf-8044-3949c8827beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:30.928988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:30.961627 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459a8d0-ca9d-4ddf-8044-3949c8827beb admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:30.965942 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fd3d5-1203-4185-a1da-b64bd1fba5bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:30.966638 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fd3d5-1203-4185-a1da-b64bd1fba5bf service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:30.967052 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fd3d5-1203-4185-a1da-b64bd1fba5bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:30.967330 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fd3d5-1203-4185-a1da-b64bd1fba5bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:30.976443 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85fd3d5-1203-4185-a1da-b64bd1fba5bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:30.978562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85fd3d5-1203-4185-a1da-b64bd1fba5bf service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:30.988067 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1376/2799] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:31.255536 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0972d-88d3-4925-90ae-ed921efc61b4 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:31.256988 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-79f0972d-88d3-4925-90ae-ed921efc61b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:31.269858 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0972d-88d3-4925-90ae-ed921efc61b4 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:31.275180 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b197c-376a-44d5-921e-76e82a6a5e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:31.276053 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b197c-376a-44d5-921e-76e82a6a5e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:31.276341 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b197c-376a-44d5-921e-76e82a6a5e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:31.276621 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b197c-376a-44d5-921e-76e82a6a5e27 admin admin] PATH_INFO: `/v3/projects/9df9bfc93ea34c3892ef94ceeebbf54b` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:31.277819 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b197c-376a-44d5-921e-76e82a6a5e27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9df9bfc93ea34c3892ef94ceeebbf54b)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:31.279780 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b197c-376a-44d5-921e-76e82a6a5e27 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:31.326639 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-842b197c-376a-44d5-921e-76e82a6a5e27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9df9bfc93ea34c3892ef94ceeebbf54b', 'request_id': 'req-842b197c-376a-44d5-921e-76e82a6a5e27'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:31.333487 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-842b197c-376a-44d5-921e-76e82a6a5e27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9df9bfc93ea34c3892ef94ceeebbf54b', 'request_id': 'req-842b197c-376a-44d5-921e-76e82a6a5e27'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:31.340428 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-842b197c-376a-44d5-921e-76e82a6a5e27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9df9bfc93ea34c3892ef94ceeebbf54b', 'request_id': 'req-842b197c-376a-44d5-921e-76e82a6a5e27'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:31.350729 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-842b197c-376a-44d5-921e-76e82a6a5e27 admin admin] The token cache is being invalidate because project 9df9bfc93ea34c3892ef94ceeebbf54b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:50:31.351016 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-842b197c-376a-44d5-921e-76e82a6a5e27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:31.351646 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-842b197c-376a-44d5-921e-76e82a6a5e27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:31.353201 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1424/2800] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:50:31 2022] DELETE /identity/v3/projects/9df9bfc93ea34c3892ef94ceeebbf54b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:31.357176 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fd3d5-1203-4185-a1da-b64bd1fba5bf service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:31.367831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fd3d5-1203-4185-a1da-b64bd1fba5bf service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:31.426705 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97cbc3-13e0-4e39-bf25-770f35b650c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:31.427584 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97cbc3-13e0-4e39-bf25-770f35b650c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:31.427864 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97cbc3-13e0-4e39-bf25-770f35b650c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:31.428137 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97cbc3-13e0-4e39-bf25-770f35b650c6 admin admin] PATH_INFO: `/v3/users/11d8b2dfba394b91b79d3b6a74bcb8ad` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:31.429307 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97cbc3-13e0-4e39-bf25-770f35b650c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11d8b2dfba394b91b79d3b6a74bcb8ad)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:31.431744 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97cbc3-13e0-4e39-bf25-770f35b650c6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:31.487756 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9c97cbc3-13e0-4e39-bf25-770f35b650c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11d8b2dfba394b91b79d3b6a74bcb8ad'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:31.504645 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9c97cbc3-13e0-4e39-bf25-770f35b650c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11d8b2dfba394b91b79d3b6a74bcb8ad'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:31.514886 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9c97cbc3-13e0-4e39-bf25-770f35b650c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11d8b2dfba394b91b79d3b6a74bcb8ad'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:31.515723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9c97cbc3-13e0-4e39-bf25-770f35b650c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11d8b2dfba394b91b79d3b6a74bcb8ad'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:31.523304 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9c97cbc3-13e0-4e39-bf25-770f35b650c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11d8b2dfba394b91b79d3b6a74bcb8ad'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:31.530036 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1377/2801] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:50:31 2022] DELETE /identity/v3/users/11d8b2dfba394b91b79d3b6a74bcb8ad => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:31.795467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b197c-376a-44d5-921e-76e82a6a5e27 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:31.796944 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-842b197c-376a-44d5-921e-76e82a6a5e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:31.811198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b197c-376a-44d5-921e-76e82a6a5e27 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:31.855065 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:31.872484 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee84d36-6ac5-4d2a-8e3a-f2996730e8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:31.873352 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee84d36-6ac5-4d2a-8e3a-f2996730e8e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:31.873635 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee84d36-6ac5-4d2a-8e3a-f2996730e8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:31.873912 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee84d36-6ac5-4d2a-8e3a-f2996730e8e5 admin admin] PATH_INFO: `/v3/projects/ee4657cc40204884baaeb3a30f11d322` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:31.875113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee84d36-6ac5-4d2a-8e3a-f2996730e8e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee4657cc40204884baaeb3a30f11d322)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:31.877352 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee84d36-6ac5-4d2a-8e3a-f2996730e8e5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:31.926333 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-bee84d36-6ac5-4d2a-8e3a-f2996730e8e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee4657cc40204884baaeb3a30f11d322', 'request_id': 'req-bee84d36-6ac5-4d2a-8e3a-f2996730e8e5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:31.934200 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-bee84d36-6ac5-4d2a-8e3a-f2996730e8e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee4657cc40204884baaeb3a30f11d322', 'request_id': 'req-bee84d36-6ac5-4d2a-8e3a-f2996730e8e5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:31.940350 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-bee84d36-6ac5-4d2a-8e3a-f2996730e8e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee4657cc40204884baaeb3a30f11d322', 'request_id': 'req-bee84d36-6ac5-4d2a-8e3a-f2996730e8e5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:31.950724 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-bee84d36-6ac5-4d2a-8e3a-f2996730e8e5 admin admin] The token cache is being invalidate because project ee4657cc40204884baaeb3a30f11d322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:50:31.951017 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-bee84d36-6ac5-4d2a-8e3a-f2996730e8e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:31.951655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-bee84d36-6ac5-4d2a-8e3a-f2996730e8e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:31.953204 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1425/2802] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:50:31 2022] DELETE /identity/v3/projects/ee4657cc40204884baaeb3a30f11d322 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:31.957995 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a13587-494c-4cf1-8f5d-160a4fb86a09 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:31.958512 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a13587-494c-4cf1-8f5d-160a4fb86a09 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:31.958664 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a13587-494c-4cf1-8f5d-160a4fb86a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:31.976109 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d5a13587-494c-4cf1-8f5d-160a4fb86a09 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:31.978018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d5a13587-494c-4cf1-8f5d-160a4fb86a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:32.019734 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:32.041812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1378/2803] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:50:32.050608 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f46023-b77f-4e04-bf42-de8ab7de57eb None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:32.050904 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f46023-b77f-4e04-bf42-de8ab7de57eb None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:32.051185 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f46023-b77f-4e04-bf42-de8ab7de57eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:32.070312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a3f46023-b77f-4e04-bf42-de8ab7de57eb None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:32.071557 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a3f46023-b77f-4e04-bf42-de8ab7de57eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:32.079585 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1426/2804] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:32.083726 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a13587-494c-4cf1-8f5d-160a4fb86a09 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:32.094065 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a13587-494c-4cf1-8f5d-160a4fb86a09 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:32.099393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ccce4-0025-4131-a335-f2fb5f822cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:32.100261 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ccce4-0025-4131-a335-f2fb5f822cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:32.100390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ccce4-0025-4131-a335-f2fb5f822cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:32.100595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ccce4-0025-4131-a335-f2fb5f822cf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:32.101462 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ccce4-0025-4131-a335-f2fb5f822cf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:50:32.101773 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ccce4-0025-4131-a335-f2fb5f822cf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:32.103951 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ccce4-0025-4131-a335-f2fb5f822cf4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:32.115846 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1379/2805] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:50:32.120803 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f46023-b77f-4e04-bf42-de8ab7de57eb None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:32.130827 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f46023-b77f-4e04-bf42-de8ab7de57eb None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:32.135613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125bdce0-558a-4e7b-9fad-61351aa9985e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:32.136264 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bdce0-558a-4e7b-9fad-61351aa9985e admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:32.136547 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bdce0-558a-4e7b-9fad-61351aa9985e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:32.136836 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bdce0-558a-4e7b-9fad-61351aa9985e admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:32.137544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125bdce0-558a-4e7b-9fad-61351aa9985e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:32.139479 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125bdce0-558a-4e7b-9fad-61351aa9985e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:32.168230 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1427/2806] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:50:32 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:32.172271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ccce4-0025-4131-a335-f2fb5f822cf4 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:32.182961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ccce4-0025-4131-a335-f2fb5f822cf4 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:32.244140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa3774-1daa-4ef6-83e2-34bf2bcc62c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:32.244816 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa3774-1daa-4ef6-83e2-34bf2bcc62c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:32.245098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa3774-1daa-4ef6-83e2-34bf2bcc62c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:32.245393 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa3774-1daa-4ef6-83e2-34bf2bcc62c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:32.246124 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa3774-1daa-4ef6-83e2-34bf2bcc62c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:32.248045 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa3774-1daa-4ef6-83e2-34bf2bcc62c5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:32.274295 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1380/2807] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:50:32 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:50:32.276015 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125bdce0-558a-4e7b-9fad-61351aa9985e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:32.288947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125bdce0-558a-4e7b-9fad-61351aa9985e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:32.294549 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d646356-013b-4106-94a4-d0cf63bc7e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:32.295093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d646356-013b-4106-94a4-d0cf63bc7e24 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:32.295381 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d646356-013b-4106-94a4-d0cf63bc7e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:32.295655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d646356-013b-4106-94a4-d0cf63bc7e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:32.296381 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d646356-013b-4106-94a4-d0cf63bc7e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:32.298082 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d646356-013b-4106-94a4-d0cf63bc7e24 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:32.309882 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1428/2808] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:32.313852 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa3774-1daa-4ef6-83e2-34bf2bcc62c5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:32.325120 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa3774-1daa-4ef6-83e2-34bf2bcc62c5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:32.329860 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c691d2-aa39-49f6-a977-7b7ce25d5889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:32.330550 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c691d2-aa39-49f6-a977-7b7ce25d5889 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:32.330831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c691d2-aa39-49f6-a977-7b7ce25d5889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:32.331105 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c691d2-aa39-49f6-a977-7b7ce25d5889 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:32.331828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c691d2-aa39-49f6-a977-7b7ce25d5889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:32.333395 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c691d2-aa39-49f6-a977-7b7ce25d5889 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:32.344105 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1381/2809] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:50:32.350328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d646356-013b-4106-94a4-d0cf63bc7e24 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:32.361261 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d646356-013b-4106-94a4-d0cf63bc7e24 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:32.366110 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680215d-4df2-4cef-a50b-e193f546cfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:32.366999 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680215d-4df2-4cef-a50b-e193f546cfde admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:32.367407 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680215d-4df2-4cef-a50b-e193f546cfde admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:32.367694 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680215d-4df2-4cef-a50b-e193f546cfde admin admin] PATH_INFO: `/v3/projects/88b59d5ac72b4d16a2d682de54c3a240/users/6f84bd63315f46079db21a4cefd17162/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:32.383804 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680215d-4df2-4cef-a50b-e193f546cfde admin admin] RBAC: Authorizing `identity:create_grant(project_id=88b59d5ac72b4d16a2d682de54c3a240, user_id=6f84bd63315f46079db21a4cefd17162, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:32.386498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680215d-4df2-4cef-a50b-e193f546cfde admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:32.400765 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1429/2810] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:50:32 2022] PUT /identity/v3/projects/88b59d5ac72b4d16a2d682de54c3a240/users/6f84bd63315f46079db21a4cefd17162/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:32.405976 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a5e92-097d-467b-ba8e-3ca82697eece None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:32.406404 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a5e92-097d-467b-ba8e-3ca82697eece None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:32.406689 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a5e92-097d-467b-ba8e-3ca82697eece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:32.432899 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-939a5e92-097d-467b-ba8e-3ca82697eece None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:32.434154 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-939a5e92-097d-467b-ba8e-3ca82697eece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:32.494228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1382/2811] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:32.504155 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680215d-4df2-4cef-a50b-e193f546cfde admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:32.505822 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6680215d-4df2-4cef-a50b-e193f546cfde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:32.563931 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680215d-4df2-4cef-a50b-e193f546cfde admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:32.567644 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6db0a4-6d25-480c-b463-6f2832e434e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:32.568298 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6db0a4-6d25-480c-b463-6f2832e434e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:32.568571 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6db0a4-6d25-480c-b463-6f2832e434e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:32.568847 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6db0a4-6d25-480c-b463-6f2832e434e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:32.578489 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6db0a4-6d25-480c-b463-6f2832e434e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:32.579927 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6db0a4-6d25-480c-b463-6f2832e434e3 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:32.589858 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1430/2812] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:33.214914 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c10406-1b60-4d31-9f42-97a4370ebb66 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:33.215576 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c10406-1b60-4d31-9f42-97a4370ebb66 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:33.215576 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c10406-1b60-4d31-9f42-97a4370ebb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:33.241041 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-36c10406-1b60-4d31-9f42-97a4370ebb66 None None] MFA Rules not processed for user `6f84bd63315f46079db21a4cefd17162`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:33.242276 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-36c10406-1b60-4d31-9f42-97a4370ebb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:33.286859 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:33.312839 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1383/2813] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:33.317244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6db0a4-6d25-480c-b463-6f2832e434e3 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:33.326772 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6db0a4-6d25-480c-b463-6f2832e434e3 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:33.331255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fddc8-86a3-4457-a508-752661bb0a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:33.331913 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fddc8-86a3-4457-a508-752661bb0a4d admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:33.332194 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fddc8-86a3-4457-a508-752661bb0a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:33.332471 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fddc8-86a3-4457-a508-752661bb0a4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:33.333166 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1fddc8-86a3-4457-a508-752661bb0a4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:33.335100 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1fddc8-86a3-4457-a508-752661bb0a4d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:33.345007 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:33.364603 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1431/2814] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:50:33 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:33.369511 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c10406-1b60-4d31-9f42-97a4370ebb66 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:33.382193 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c10406-1b60-4d31-9f42-97a4370ebb66 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:33.439048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a7d80-ef56-458b-8bd0-0e689c038308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:33.439719 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a7d80-ef56-458b-8bd0-0e689c038308 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:33.440005 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a7d80-ef56-458b-8bd0-0e689c038308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:33.440283 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a7d80-ef56-458b-8bd0-0e689c038308 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:33.440981 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a7d80-ef56-458b-8bd0-0e689c038308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:33.442984 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a7d80-ef56-458b-8bd0-0e689c038308 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:33.461539 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1384/2815] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:50:33 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:50:33.465582 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fddc8-86a3-4457-a508-752661bb0a4d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:33.476083 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fddc8-86a3-4457-a508-752661bb0a4d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:33.481379 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60678d-c57a-4ed0-849f-eb49b33b5b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:33.482069 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60678d-c57a-4ed0-849f-eb49b33b5b2a admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:33.482355 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60678d-c57a-4ed0-849f-eb49b33b5b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:33.482652 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60678d-c57a-4ed0-849f-eb49b33b5b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:33.483528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a60678d-c57a-4ed0-849f-eb49b33b5b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:33.485123 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a60678d-c57a-4ed0-849f-eb49b33b5b2a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:33.499869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1432/2816] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:33.501008 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a7d80-ef56-458b-8bd0-0e689c038308 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:33.511616 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a7d80-ef56-458b-8bd0-0e689c038308 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:33.516595 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4287c96-1886-4caf-a47b-53cd84c52ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:33.517569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4287c96-1886-4caf-a47b-53cd84c52ef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:33.517909 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4287c96-1886-4caf-a47b-53cd84c52ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:33.518226 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4287c96-1886-4caf-a47b-53cd84c52ef5 admin admin] PATH_INFO: `/v3/projects/3000a4da211e46639001c8d2cfc20211/users/52715cb3faea47098ab7c8268eb9a85b/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:33.533804 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4287c96-1886-4caf-a47b-53cd84c52ef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3000a4da211e46639001c8d2cfc20211, user_id=52715cb3faea47098ab7c8268eb9a85b, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:33.536508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4287c96-1886-4caf-a47b-53cd84c52ef5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:33.553070 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1385/2817] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:50:33 2022] PUT /identity/v3/projects/3000a4da211e46639001c8d2cfc20211/users/52715cb3faea47098ab7c8268eb9a85b/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:50:33.557565 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60678d-c57a-4ed0-849f-eb49b33b5b2a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:33.568070 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60678d-c57a-4ed0-849f-eb49b33b5b2a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:33.624782 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f520d-19da-4b70-acc4-e3675ab89256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:33.625409 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f520d-19da-4b70-acc4-e3675ab89256 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:33.625694 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f520d-19da-4b70-acc4-e3675ab89256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:33.625983 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f520d-19da-4b70-acc4-e3675ab89256 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:33.626757 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f520d-19da-4b70-acc4-e3675ab89256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:33.628619 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f520d-19da-4b70-acc4-e3675ab89256 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:33.638604 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1433/2818] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:33.642870 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4287c96-1886-4caf-a47b-53cd84c52ef5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:33.653753 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4287c96-1886-4caf-a47b-53cd84c52ef5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:33.658409 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b560b-bfdc-46e4-bb36-4b1651001faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:33.659283 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b560b-bfdc-46e4-bb36-4b1651001faa admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:33.659571 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b560b-bfdc-46e4-bb36-4b1651001faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:33.659867 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b560b-bfdc-46e4-bb36-4b1651001faa admin admin] PATH_INFO: `/v3/projects/3000a4da211e46639001c8d2cfc20211/users/52715cb3faea47098ab7c8268eb9a85b/roles/864a6966630544e8bd68bcafba905c6f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:33.661856 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b560b-bfdc-46e4-bb36-4b1651001faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3000a4da211e46639001c8d2cfc20211, user_id=52715cb3faea47098ab7c8268eb9a85b, role_id=864a6966630544e8bd68bcafba905c6f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:33.664648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b560b-bfdc-46e4-bb36-4b1651001faa admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:33.677500 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1386/2819] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:50:33 2022] PUT /identity/v3/projects/3000a4da211e46639001c8d2cfc20211/users/52715cb3faea47098ab7c8268eb9a85b/roles/864a6966630544e8bd68bcafba905c6f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:34.343677 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce868257-50b6-4790-9735-b764f74b4914 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:34.344613 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce868257-50b6-4790-9735-b764f74b4914 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:34.345062 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce868257-50b6-4790-9735-b764f74b4914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:34.402277 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ce868257-50b6-4790-9735-b764f74b4914 None None] MFA Rules not processed for user `52715cb3faea47098ab7c8268eb9a85b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:34.404104 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ce868257-50b6-4790-9735-b764f74b4914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:34.488111 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1434/2820] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:34.499934 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b560b-bfdc-46e4-bb36-4b1651001faa admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:34.501744 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4f7b560b-bfdc-46e4-bb36-4b1651001faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:34.563019 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b560b-bfdc-46e4-bb36-4b1651001faa admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:34.566661 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd74c1-6cc1-4216-954c-1419d7635e69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:34.567305 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd74c1-6cc1-4216-954c-1419d7635e69 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:34.567590 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd74c1-6cc1-4216-954c-1419d7635e69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:34.567868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd74c1-6cc1-4216-954c-1419d7635e69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:34.611399 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:34.624175 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd74c1-6cc1-4216-954c-1419d7635e69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:34.625619 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd74c1-6cc1-4216-954c-1419d7635e69 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:34.635252 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1387/2821] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:35.886823 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b822de4-b679-400c-bd42-2a01f4ea4b3b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:35.887255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b822de4-b679-400c-bd42-2a01f4ea4b3b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:35.887369 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b822de4-b679-400c-bd42-2a01f4ea4b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:35.915953 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5b822de4-b679-400c-bd42-2a01f4ea4b3b None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:35.917458 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5b822de4-b679-400c-bd42-2a01f4ea4b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:35.964551 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:35.974911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1435/2822] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jul 12 13:50:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:36.006727 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd74c1-6cc1-4216-954c-1419d7635e69 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:36.008146 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cccd74c1-6cc1-4216-954c-1419d7635e69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:36.020963 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd74c1-6cc1-4216-954c-1419d7635e69 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:36.023028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b822de4-b679-400c-bd42-2a01f4ea4b3b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:36.024976 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5b822de4-b679-400c-bd42-2a01f4ea4b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:36.025264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe7a20-985f-465e-ba30-540546a0cc2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:36.025913 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe7a20-985f-465e-ba30-540546a0cc2f service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:36.026228 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe7a20-985f-465e-ba30-540546a0cc2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:36.026547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe7a20-985f-465e-ba30-540546a0cc2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:36.035732 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe7a20-985f-465e-ba30-540546a0cc2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:36.037153 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe7a20-985f-465e-ba30-540546a0cc2f service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:36.047268 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1388/2823] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:36.094968 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b822de4-b679-400c-bd42-2a01f4ea4b3b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:36.100391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6e776-5986-439e-8074-b95e55ccf48d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:36.101048 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6e776-5986-439e-8074-b95e55ccf48d service swift] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:36.101336 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6e776-5986-439e-8074-b95e55ccf48d service swift] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:36.101617 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6e776-5986-439e-8074-b95e55ccf48d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:36.111042 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb6e776-5986-439e-8074-b95e55ccf48d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:36.112892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb6e776-5986-439e-8074-b95e55ccf48d service swift] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:36.121681 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1436/2824] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:50:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 13:50:36.638002 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa94a0-4554-49b3-a21b-eb3e9ec62d6e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:36.638300 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa94a0-4554-49b3-a21b-eb3e9ec62d6e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:36.638581 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa94a0-4554-49b3-a21b-eb3e9ec62d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:36.655861 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9bfa94a0-4554-49b3-a21b-eb3e9ec62d6e None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:36.657046 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9bfa94a0-4554-49b3-a21b-eb3e9ec62d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:36.664218 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1389/2825] 10.222.0.52 () {58 vars in 967 bytes} [Tue Jul 12 13:50:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:36.677940 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6e776-5986-439e-8074-b95e55ccf48d service swift] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:36.688848 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6e776-5986-439e-8074-b95e55ccf48d service swift] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:36.693908 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d92ccf-87a8-42e9-9501-039b4d64ab2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:36.694600 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d92ccf-87a8-42e9-9501-039b4d64ab2a service swift] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:36.694882 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d92ccf-87a8-42e9-9501-039b4d64ab2a service swift] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:36.695159 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d92ccf-87a8-42e9-9501-039b4d64ab2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:36.703910 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d92ccf-87a8-42e9-9501-039b4d64ab2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:36.706921 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d92ccf-87a8-42e9-9501-039b4d64ab2a service swift] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:36.719857 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1437/2826] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:50:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:50:36.905521 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ce4cc-ec98-4e97-a4c2-e60e186f95c5 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:36.905817 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ce4cc-ec98-4e97-a4c2-e60e186f95c5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:36.906140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ce4cc-ec98-4e97-a4c2-e60e186f95c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:36.924579 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-815ce4cc-ec98-4e97-a4c2-e60e186f95c5 None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:36.925806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-815ce4cc-ec98-4e97-a4c2-e60e186f95c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:36.934345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1390/2827] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jul 12 13:50:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:36.946210 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d92ccf-87a8-42e9-9501-039b4d64ab2a service swift] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:36.956777 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d92ccf-87a8-42e9-9501-039b4d64ab2a service swift] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:36.962483 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2cdd4-9310-4354-b22f-5a5067664f8a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:36.963113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2cdd4-9310-4354-b22f-5a5067664f8a service swift] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:36.963395 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2cdd4-9310-4354-b22f-5a5067664f8a service swift] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:36.963672 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2cdd4-9310-4354-b22f-5a5067664f8a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:36.971693 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2cdd4-9310-4354-b22f-5a5067664f8a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:36.973273 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2cdd4-9310-4354-b22f-5a5067664f8a service swift] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:36.981785 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1438/2828] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:50:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:50:37.292185 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e53582-0b66-4c53-a8e6-375597090b90 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:37.292624 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e53582-0b66-4c53-a8e6-375597090b90 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:37.292723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e53582-0b66-4c53-a8e6-375597090b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:37.313797 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a0e53582-0b66-4c53-a8e6-375597090b90 None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:37.315178 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a0e53582-0b66-4c53-a8e6-375597090b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:37.323194 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1391/2829] 10.222.0.52 () {58 vars in 967 bytes} [Tue Jul 12 13:50:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:37.335332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2cdd4-9310-4354-b22f-5a5067664f8a service swift] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:37.346757 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2cdd4-9310-4354-b22f-5a5067664f8a service swift] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:37.351895 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228da0d-b876-4c3c-a596-eaaf2642b34a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:37.352532 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228da0d-b876-4c3c-a596-eaaf2642b34a service swift] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:37.352856 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228da0d-b876-4c3c-a596-eaaf2642b34a service swift] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:37.353139 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228da0d-b876-4c3c-a596-eaaf2642b34a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:37.361455 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228da0d-b876-4c3c-a596-eaaf2642b34a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:37.363555 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228da0d-b876-4c3c-a596-eaaf2642b34a service swift] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:37.371801 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1439/2830] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:50:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:50:39.432775 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e53582-0b66-4c53-a8e6-375597090b90 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:39.434290 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a0e53582-0b66-4c53-a8e6-375597090b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:39.498624 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e53582-0b66-4c53-a8e6-375597090b90 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:39.503412 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41ba33-ef60-4391-a0be-f4f16a37c444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:39.504284 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41ba33-ef60-4391-a0be-f4f16a37c444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:39.504569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41ba33-ef60-4391-a0be-f4f16a37c444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:39.504847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41ba33-ef60-4391-a0be-f4f16a37c444 admin admin] PATH_INFO: `/v3/users/0c30e76da528404d964782e701cefce2` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:39.506078 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41ba33-ef60-4391-a0be-f4f16a37c444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c30e76da528404d964782e701cefce2)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:39.507993 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41ba33-ef60-4391-a0be-f4f16a37c444 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:39.552178 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1c41ba33-ef60-4391-a0be-f4f16a37c444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c30e76da528404d964782e701cefce2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:39.564270 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1c41ba33-ef60-4391-a0be-f4f16a37c444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c30e76da528404d964782e701cefce2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:39.570895 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1c41ba33-ef60-4391-a0be-f4f16a37c444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c30e76da528404d964782e701cefce2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:39.572044 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1c41ba33-ef60-4391-a0be-f4f16a37c444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c30e76da528404d964782e701cefce2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:39.578744 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1c41ba33-ef60-4391-a0be-f4f16a37c444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c30e76da528404d964782e701cefce2'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:39.584626 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1392/2831] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:50:39 2022] DELETE /identity/v3/users/0c30e76da528404d964782e701cefce2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:39.589708 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eb233-5781-40ea-baa6-07d1570999ad None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:39.590189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eb233-5781-40ea-baa6-07d1570999ad None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:39.590309 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eb233-5781-40ea-baa6-07d1570999ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:39.607588 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-268eb233-5781-40ea-baa6-07d1570999ad None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:39.608787 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-268eb233-5781-40ea-baa6-07d1570999ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:39.673757 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1440/2832] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:39.683724 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41ba33-ef60-4391-a0be-f4f16a37c444 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:39.685117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1c41ba33-ef60-4391-a0be-f4f16a37c444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:39.720190 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:39.743234 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41ba33-ef60-4391-a0be-f4f16a37c444 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:39.747387 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62226674-0e9f-42b2-8c7d-31ef1d1cd6e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:39.748023 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62226674-0e9f-42b2-8c7d-31ef1d1cd6e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:39.748313 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62226674-0e9f-42b2-8c7d-31ef1d1cd6e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:39.748590 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62226674-0e9f-42b2-8c7d-31ef1d1cd6e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:39.757834 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62226674-0e9f-42b2-8c7d-31ef1d1cd6e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:39.759369 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62226674-0e9f-42b2-8c7d-31ef1d1cd6e1 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:39.769487 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1393/2833] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:39.941852 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268eb233-5781-40ea-baa6-07d1570999ad None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:39.943403 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-268eb233-5781-40ea-baa6-07d1570999ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:39.965986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268eb233-5781-40ea-baa6-07d1570999ad None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:39.970939 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15cb578-0023-4e4f-9a92-7fa34622822f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:39.971845 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cb578-0023-4e4f-9a92-7fa34622822f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:39.972674 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cb578-0023-4e4f-9a92-7fa34622822f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:39.972961 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cb578-0023-4e4f-9a92-7fa34622822f admin admin] PATH_INFO: `/v3/projects/8156e0a1827642ef828e8b88343f088d` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:39.975033 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15cb578-0023-4e4f-9a92-7fa34622822f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8156e0a1827642ef828e8b88343f088d)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:39.977787 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15cb578-0023-4e4f-9a92-7fa34622822f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:40.030796 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:40.037093 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d15cb578-0023-4e4f-9a92-7fa34622822f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8156e0a1827642ef828e8b88343f088d', 'request_id': 'req-d15cb578-0023-4e4f-9a92-7fa34622822f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:40.046354 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d15cb578-0023-4e4f-9a92-7fa34622822f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8156e0a1827642ef828e8b88343f088d', 'request_id': 'req-d15cb578-0023-4e4f-9a92-7fa34622822f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:40.053195 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d15cb578-0023-4e4f-9a92-7fa34622822f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8156e0a1827642ef828e8b88343f088d', 'request_id': 'req-d15cb578-0023-4e4f-9a92-7fa34622822f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:40.065275 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d15cb578-0023-4e4f-9a92-7fa34622822f admin admin] The token cache is being invalidate because project 8156e0a1827642ef828e8b88343f088d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:50:40.065498 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d15cb578-0023-4e4f-9a92-7fa34622822f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:40.066170 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d15cb578-0023-4e4f-9a92-7fa34622822f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:40.067725 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1441/2834] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:50:39 2022] DELETE /identity/v3/projects/8156e0a1827642ef828e8b88343f088d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:48.947534 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62226674-0e9f-42b2-8c7d-31ef1d1cd6e1 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:48.949015 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-62226674-0e9f-42b2-8c7d-31ef1d1cd6e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:48.963310 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62226674-0e9f-42b2-8c7d-31ef1d1cd6e1 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:49.018107 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18d9ef-66da-4bed-8c47-d6e32121a52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:49.018985 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18d9ef-66da-4bed-8c47-d6e32121a52b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:49.019270 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18d9ef-66da-4bed-8c47-d6e32121a52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:49.019543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18d9ef-66da-4bed-8c47-d6e32121a52b admin admin] PATH_INFO: `/v3/users/6f84bd63315f46079db21a4cefd17162` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:49.020738 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18d9ef-66da-4bed-8c47-d6e32121a52b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f84bd63315f46079db21a4cefd17162)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:49.022677 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18d9ef-66da-4bed-8c47-d6e32121a52b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:49.068372 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9d18d9ef-66da-4bed-8c47-d6e32121a52b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f84bd63315f46079db21a4cefd17162'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.080625 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9d18d9ef-66da-4bed-8c47-d6e32121a52b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f84bd63315f46079db21a4cefd17162'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.087384 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9d18d9ef-66da-4bed-8c47-d6e32121a52b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f84bd63315f46079db21a4cefd17162'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.088704 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9d18d9ef-66da-4bed-8c47-d6e32121a52b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f84bd63315f46079db21a4cefd17162'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.096189 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9d18d9ef-66da-4bed-8c47-d6e32121a52b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f84bd63315f46079db21a4cefd17162'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.102407 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1394/2835] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:50:48 2022] DELETE /identity/v3/users/6f84bd63315f46079db21a4cefd17162 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:50:49.273332 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15cb578-0023-4e4f-9a92-7fa34622822f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:49.274858 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d15cb578-0023-4e4f-9a92-7fa34622822f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:49.288360 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15cb578-0023-4e4f-9a92-7fa34622822f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:49.349907 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f13d05-8a21-4a81-8c66-e945c2a65d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:49.351623 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13d05-8a21-4a81-8c66-e945c2a65d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:49.352113 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13d05-8a21-4a81-8c66-e945c2a65d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:49.352399 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13d05-8a21-4a81-8c66-e945c2a65d90 admin admin] PATH_INFO: `/v3/projects/88b59d5ac72b4d16a2d682de54c3a240` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:49.353617 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f13d05-8a21-4a81-8c66-e945c2a65d90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88b59d5ac72b4d16a2d682de54c3a240)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:49.355546 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f13d05-8a21-4a81-8c66-e945c2a65d90 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:49.402752 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b8f13d05-8a21-4a81-8c66-e945c2a65d90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88b59d5ac72b4d16a2d682de54c3a240', 'request_id': 'req-b8f13d05-8a21-4a81-8c66-e945c2a65d90'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.410118 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b8f13d05-8a21-4a81-8c66-e945c2a65d90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88b59d5ac72b4d16a2d682de54c3a240', 'request_id': 'req-b8f13d05-8a21-4a81-8c66-e945c2a65d90'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.416219 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b8f13d05-8a21-4a81-8c66-e945c2a65d90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88b59d5ac72b4d16a2d682de54c3a240', 'request_id': 'req-b8f13d05-8a21-4a81-8c66-e945c2a65d90'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.426415 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b8f13d05-8a21-4a81-8c66-e945c2a65d90 admin admin] The token cache is being invalidate because project 88b59d5ac72b4d16a2d682de54c3a240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:50:49.426700 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b8f13d05-8a21-4a81-8c66-e945c2a65d90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.427326 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b8f13d05-8a21-4a81-8c66-e945c2a65d90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.428874 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1442/2836] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:50:49 2022] DELETE /identity/v3/projects/88b59d5ac72b4d16a2d682de54c3a240 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:49.433553 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18d9ef-66da-4bed-8c47-d6e32121a52b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:49.434986 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9d18d9ef-66da-4bed-8c47-d6e32121a52b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:49.447612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18d9ef-66da-4bed-8c47-d6e32121a52b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:49.495752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:49.503713 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660463ac-e4c9-45f4-aec8-4a10dc46b824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:49.504828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660463ac-e4c9-45f4-aec8-4a10dc46b824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:49.505331 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660463ac-e4c9-45f4-aec8-4a10dc46b824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:49.505802 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660463ac-e4c9-45f4-aec8-4a10dc46b824 admin admin] PATH_INFO: `/v3/users/52715cb3faea47098ab7c8268eb9a85b` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:49.507089 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660463ac-e4c9-45f4-aec8-4a10dc46b824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52715cb3faea47098ab7c8268eb9a85b)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:49.509001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660463ac-e4c9-45f4-aec8-4a10dc46b824 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:49.554269 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-660463ac-e4c9-45f4-aec8-4a10dc46b824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52715cb3faea47098ab7c8268eb9a85b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.566493 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-660463ac-e4c9-45f4-aec8-4a10dc46b824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52715cb3faea47098ab7c8268eb9a85b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.573807 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-660463ac-e4c9-45f4-aec8-4a10dc46b824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52715cb3faea47098ab7c8268eb9a85b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.574492 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-660463ac-e4c9-45f4-aec8-4a10dc46b824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52715cb3faea47098ab7c8268eb9a85b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.581673 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-660463ac-e4c9-45f4-aec8-4a10dc46b824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52715cb3faea47098ab7c8268eb9a85b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.587795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1395/2837] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:50:49 2022] DELETE /identity/v3/users/52715cb3faea47098ab7c8268eb9a85b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:49.763778 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f13d05-8a21-4a81-8c66-e945c2a65d90 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:49.765377 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b8f13d05-8a21-4a81-8c66-e945c2a65d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:49.778228 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f13d05-8a21-4a81-8c66-e945c2a65d90 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:49.833328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f280a-afea-447d-ba68-a77b65a34cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:49.834226 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f280a-afea-447d-ba68-a77b65a34cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:49.834504 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f280a-afea-447d-ba68-a77b65a34cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:49.834773 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f280a-afea-447d-ba68-a77b65a34cef admin admin] PATH_INFO: `/v3/projects/3000a4da211e46639001c8d2cfc20211` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:49.835932 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f280a-afea-447d-ba68-a77b65a34cef admin admin] RBAC: Authorizing `identity:delete_project(project_id=3000a4da211e46639001c8d2cfc20211)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:49.838004 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f280a-afea-447d-ba68-a77b65a34cef admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:49.840167 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:49.887391 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6c5f280a-afea-447d-ba68-a77b65a34cef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3000a4da211e46639001c8d2cfc20211', 'request_id': 'req-6c5f280a-afea-447d-ba68-a77b65a34cef'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.894376 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6c5f280a-afea-447d-ba68-a77b65a34cef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3000a4da211e46639001c8d2cfc20211', 'request_id': 'req-6c5f280a-afea-447d-ba68-a77b65a34cef'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.900689 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6c5f280a-afea-447d-ba68-a77b65a34cef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3000a4da211e46639001c8d2cfc20211', 'request_id': 'req-6c5f280a-afea-447d-ba68-a77b65a34cef'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.910947 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6c5f280a-afea-447d-ba68-a77b65a34cef admin admin] The token cache is being invalidate because project 3000a4da211e46639001c8d2cfc20211 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:50:49.911227 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6c5f280a-afea-447d-ba68-a77b65a34cef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.911843 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6c5f280a-afea-447d-ba68-a77b65a34cef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:49.913358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1443/2838] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:50:49 2022] DELETE /identity/v3/projects/3000a4da211e46639001c8d2cfc20211 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:49.918945 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad235d-4047-4239-97d7-726e11eaccbe None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:49.919234 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad235d-4047-4239-97d7-726e11eaccbe None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:49.919510 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad235d-4047-4239-97d7-726e11eaccbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:49.938037 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-97ad235d-4047-4239-97d7-726e11eaccbe None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:49.939157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-97ad235d-4047-4239-97d7-726e11eaccbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:49.996928 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1396/2839] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:50.005128 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070bbde-eb83-4a9c-a2f1-2cb6ac4d2eda None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:50.005421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070bbde-eb83-4a9c-a2f1-2cb6ac4d2eda None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:50.005699 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070bbde-eb83-4a9c-a2f1-2cb6ac4d2eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:50.023354 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9070bbde-eb83-4a9c-a2f1-2cb6ac4d2eda None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:50.024540 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9070bbde-eb83-4a9c-a2f1-2cb6ac4d2eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:50.032310 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1444/2840] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:50.036382 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad235d-4047-4239-97d7-726e11eaccbe None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:50.046575 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad235d-4047-4239-97d7-726e11eaccbe None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:50.051076 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc7db7-5b06-4329-85ab-52a54abf1be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:50.051714 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc7db7-5b06-4329-85ab-52a54abf1be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:50.051992 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc7db7-5b06-4329-85ab-52a54abf1be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:50.052263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc7db7-5b06-4329-85ab-52a54abf1be0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:50.052981 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc7db7-5b06-4329-85ab-52a54abf1be0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:50:50.053285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc7db7-5b06-4329-85ab-52a54abf1be0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:50.054897 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc7db7-5b06-4329-85ab-52a54abf1be0 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:50.066536 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1397/2841] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:50:50.070696 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070bbde-eb83-4a9c-a2f1-2cb6ac4d2eda None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:50.079981 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070bbde-eb83-4a9c-a2f1-2cb6ac4d2eda None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:50.084630 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22002bd-b1cb-497f-a558-0605c2d1f887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:50.085269 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22002bd-b1cb-497f-a558-0605c2d1f887 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:50.085555 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22002bd-b1cb-497f-a558-0605c2d1f887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:50.085836 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22002bd-b1cb-497f-a558-0605c2d1f887 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:50.086623 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22002bd-b1cb-497f-a558-0605c2d1f887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:50.088531 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22002bd-b1cb-497f-a558-0605c2d1f887 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:50.114050 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1445/2842] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:50:50 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:50.118083 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc7db7-5b06-4329-85ab-52a54abf1be0 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:50.128797 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc7db7-5b06-4329-85ab-52a54abf1be0 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:50.184168 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd1972-7653-4af4-9005-f3e4b6cc1ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:50.184826 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd1972-7653-4af4-9005-f3e4b6cc1ccb admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:50.185109 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd1972-7653-4af4-9005-f3e4b6cc1ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:50.185394 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd1972-7653-4af4-9005-f3e4b6cc1ccb admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:50.186452 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd1972-7653-4af4-9005-f3e4b6cc1ccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:50.188417 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd1972-7653-4af4-9005-f3e4b6cc1ccb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:50.195684 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:50.212083 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1398/2843] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:50:50 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:50.213690 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22002bd-b1cb-497f-a558-0605c2d1f887 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:50.223858 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22002bd-b1cb-497f-a558-0605c2d1f887 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:50.229072 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6dc9dc-63ea-4927-96d7-d8f0d81ac433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:50.229726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6dc9dc-63ea-4927-96d7-d8f0d81ac433 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:50.230060 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6dc9dc-63ea-4927-96d7-d8f0d81ac433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:50.230329 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6dc9dc-63ea-4927-96d7-d8f0d81ac433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:50.231057 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6dc9dc-63ea-4927-96d7-d8f0d81ac433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:50.232667 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6dc9dc-63ea-4927-96d7-d8f0d81ac433 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:50.242872 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1446/2844] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:50:50.246688 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd1972-7653-4af4-9005-f3e4b6cc1ccb admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:50.256787 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd1972-7653-4af4-9005-f3e4b6cc1ccb admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:50.261369 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48436088-c01a-4d17-8285-2a0f6a8153a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:50.262169 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48436088-c01a-4d17-8285-2a0f6a8153a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:50.262459 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48436088-c01a-4d17-8285-2a0f6a8153a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:50.262736 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48436088-c01a-4d17-8285-2a0f6a8153a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:50.263458 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48436088-c01a-4d17-8285-2a0f6a8153a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:50.265024 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48436088-c01a-4d17-8285-2a0f6a8153a5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:50.274662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1399/2845] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:50.278926 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6dc9dc-63ea-4927-96d7-d8f0d81ac433 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:50.288806 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6dc9dc-63ea-4927-96d7-d8f0d81ac433 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:50.294034 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23abd34-1ceb-4d97-b93f-17585bd44b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:50.294903 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23abd34-1ceb-4d97-b93f-17585bd44b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:50.295189 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23abd34-1ceb-4d97-b93f-17585bd44b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:50.295467 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23abd34-1ceb-4d97-b93f-17585bd44b6e admin admin] PATH_INFO: `/v3/projects/63c7829835394ad193c97be2c28ec801/users/97adb1819a214bd198d07c894b7681d5/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:50.309784 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23abd34-1ceb-4d97-b93f-17585bd44b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c7829835394ad193c97be2c28ec801, user_id=97adb1819a214bd198d07c894b7681d5, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:50.312464 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23abd34-1ceb-4d97-b93f-17585bd44b6e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:50.325285 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1447/2846] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:50:50 2022] PUT /identity/v3/projects/63c7829835394ad193c97be2c28ec801/users/97adb1819a214bd198d07c894b7681d5/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:50:50.335815 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890aa60-3d43-49d1-8b12-4a18e96d28ef None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:50.336113 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890aa60-3d43-49d1-8b12-4a18e96d28ef None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:50.336412 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890aa60-3d43-49d1-8b12-4a18e96d28ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:50.363463 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b890aa60-3d43-49d1-8b12-4a18e96d28ef None None] MFA Rules not processed for user `97adb1819a214bd198d07c894b7681d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:50.364699 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b890aa60-3d43-49d1-8b12-4a18e96d28ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:50.427227 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1400/2847] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:50.437737 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23abd34-1ceb-4d97-b93f-17585bd44b6e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:50.439478 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e23abd34-1ceb-4d97-b93f-17585bd44b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:50.494986 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23abd34-1ceb-4d97-b93f-17585bd44b6e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:50.498617 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c3f60-d8d3-4a59-ad86-23341447d319 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:50.499256 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c3f60-d8d3-4a59-ad86-23341447d319 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:50.499544 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c3f60-d8d3-4a59-ad86-23341447d319 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:50.499823 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c3f60-d8d3-4a59-ad86-23341447d319 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:50.508419 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c3f60-d8d3-4a59-ad86-23341447d319 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:50.510405 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c3f60-d8d3-4a59-ad86-23341447d319 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:50.519374 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1448/2848] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:57.830328 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890aa60-3d43-49d1-8b12-4a18e96d28ef None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:57.892635 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890aa60-3d43-49d1-8b12-4a18e96d28ef None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:57.897242 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc868b0-11ae-4f31-9170-f90c25cbc3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:57.898150 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc868b0-11ae-4f31-9170-f90c25cbc3ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:57.898435 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc868b0-11ae-4f31-9170-f90c25cbc3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:57.898711 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc868b0-11ae-4f31-9170-f90c25cbc3ea admin admin] PATH_INFO: `/v3/users/97adb1819a214bd198d07c894b7681d5` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:57.899890 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc868b0-11ae-4f31-9170-f90c25cbc3ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=97adb1819a214bd198d07c894b7681d5)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:57.901793 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc868b0-11ae-4f31-9170-f90c25cbc3ea admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:57.944824 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9cc868b0-11ae-4f31-9170-f90c25cbc3ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97adb1819a214bd198d07c894b7681d5'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:57.957568 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9cc868b0-11ae-4f31-9170-f90c25cbc3ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97adb1819a214bd198d07c894b7681d5'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:57.962830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:57.964957 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9cc868b0-11ae-4f31-9170-f90c25cbc3ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97adb1819a214bd198d07c894b7681d5'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:57.965845 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9cc868b0-11ae-4f31-9170-f90c25cbc3ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97adb1819a214bd198d07c894b7681d5'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:57.973795 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-9cc868b0-11ae-4f31-9170-f90c25cbc3ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97adb1819a214bd198d07c894b7681d5'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:57.980140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1401/2849] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:50:57 2022] DELETE /identity/v3/users/97adb1819a214bd198d07c894b7681d5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:57.984722 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f6715-c24f-483c-b5fe-091f0027fa2a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:57.985193 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f6715-c24f-483c-b5fe-091f0027fa2a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:57.985294 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f6715-c24f-483c-b5fe-091f0027fa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:58.003235 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-409f6715-c24f-483c-b5fe-091f0027fa2a None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:58.004421 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-409f6715-c24f-483c-b5fe-091f0027fa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:58.028783 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:58.064734 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1449/2850] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:58.074418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc868b0-11ae-4f31-9170-f90c25cbc3ea admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:58.075828 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9cc868b0-11ae-4f31-9170-f90c25cbc3ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:58.133143 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc868b0-11ae-4f31-9170-f90c25cbc3ea admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:58.136797 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2071b-a517-42e4-af14-072666c8ec27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:58.137432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2071b-a517-42e4-af14-072666c8ec27 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:58.137715 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2071b-a517-42e4-af14-072666c8ec27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:58.138018 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2071b-a517-42e4-af14-072666c8ec27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:58.147004 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2071b-a517-42e4-af14-072666c8ec27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:58.148426 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2071b-a517-42e4-af14-072666c8ec27 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:58.158123 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1402/2851] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:50:58.429609 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f6715-c24f-483c-b5fe-091f0027fa2a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:58.431054 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-409f6715-c24f-483c-b5fe-091f0027fa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:58.448863 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f6715-c24f-483c-b5fe-091f0027fa2a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:58.454651 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb97f07-317d-4acd-b07d-7288882d7e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:58.455521 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb97f07-317d-4acd-b07d-7288882d7e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:58.455810 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb97f07-317d-4acd-b07d-7288882d7e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:58.456090 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb97f07-317d-4acd-b07d-7288882d7e5e admin admin] PATH_INFO: `/v3/projects/63c7829835394ad193c97be2c28ec801` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:58.457308 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb97f07-317d-4acd-b07d-7288882d7e5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=63c7829835394ad193c97be2c28ec801)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:58.459268 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb97f07-317d-4acd-b07d-7288882d7e5e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:58.507162 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-0bb97f07-317d-4acd-b07d-7288882d7e5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63c7829835394ad193c97be2c28ec801', 'request_id': 'req-0bb97f07-317d-4acd-b07d-7288882d7e5e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:58.514578 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-0bb97f07-317d-4acd-b07d-7288882d7e5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63c7829835394ad193c97be2c28ec801', 'request_id': 'req-0bb97f07-317d-4acd-b07d-7288882d7e5e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:58.520815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-0bb97f07-317d-4acd-b07d-7288882d7e5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63c7829835394ad193c97be2c28ec801', 'request_id': 'req-0bb97f07-317d-4acd-b07d-7288882d7e5e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:58.531626 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-0bb97f07-317d-4acd-b07d-7288882d7e5e admin admin] The token cache is being invalidate because project 63c7829835394ad193c97be2c28ec801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:50:58.531918 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-0bb97f07-317d-4acd-b07d-7288882d7e5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:58.532541 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-0bb97f07-317d-4acd-b07d-7288882d7e5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:50:58.534094 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1450/2852] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:50:58 2022] DELETE /identity/v3/projects/63c7829835394ad193c97be2c28ec801 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:58.539271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040047e-fbe4-4ef0-8245-021814e7a54b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:58.539567 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040047e-fbe4-4ef0-8245-021814e7a54b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:58.539858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040047e-fbe4-4ef0-8245-021814e7a54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:58.557601 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e040047e-fbe4-4ef0-8245-021814e7a54b None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:58.558877 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e040047e-fbe4-4ef0-8245-021814e7a54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:58.616806 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1403/2853] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:58.679258 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643cf8a-1331-4283-af30-93f96d799142 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:58.679532 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643cf8a-1331-4283-af30-93f96d799142 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:58.679817 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643cf8a-1331-4283-af30-93f96d799142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:58.697818 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b643cf8a-1331-4283-af30-93f96d799142 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:58.699058 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b643cf8a-1331-4283-af30-93f96d799142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:58.706996 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1451/2854] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:58.711392 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040047e-fbe4-4ef0-8245-021814e7a54b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:58.721578 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040047e-fbe4-4ef0-8245-021814e7a54b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:58.726183 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27188e-f0b3-4714-9ac2-ad760b0a2c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:58.726917 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27188e-f0b3-4714-9ac2-ad760b0a2c19 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:58.727200 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27188e-f0b3-4714-9ac2-ad760b0a2c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:58.727478 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27188e-f0b3-4714-9ac2-ad760b0a2c19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:58.728805 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27188e-f0b3-4714-9ac2-ad760b0a2c19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:50:58.729111 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27188e-f0b3-4714-9ac2-ad760b0a2c19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:58.730778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27188e-f0b3-4714-9ac2-ad760b0a2c19 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:58.742005 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1404/2855] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:50:58.746067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643cf8a-1331-4283-af30-93f96d799142 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:58.755517 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643cf8a-1331-4283-af30-93f96d799142 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:58.760719 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4569cce-33ba-4207-b8c8-798df2f15522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:58.761358 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4569cce-33ba-4207-b8c8-798df2f15522 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:58.761641 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4569cce-33ba-4207-b8c8-798df2f15522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:58.761918 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4569cce-33ba-4207-b8c8-798df2f15522 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:58.762659 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4569cce-33ba-4207-b8c8-798df2f15522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:58.764556 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4569cce-33ba-4207-b8c8-798df2f15522 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:58.789299 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1452/2856] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:50:58 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:58.793352 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27188e-f0b3-4714-9ac2-ad760b0a2c19 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:58.803225 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27188e-f0b3-4714-9ac2-ad760b0a2c19 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:58.864992 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2e948-29a8-44c9-b96e-57d3c46319b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:58.865662 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2e948-29a8-44c9-b96e-57d3c46319b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:58.865946 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2e948-29a8-44c9-b96e-57d3c46319b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:58.866263 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2e948-29a8-44c9-b96e-57d3c46319b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:58.867012 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2e948-29a8-44c9-b96e-57d3c46319b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:58.868915 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2e948-29a8-44c9-b96e-57d3c46319b2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:58.888101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1405/2857] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:50:58 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:50:58.892324 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4569cce-33ba-4207-b8c8-798df2f15522 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:58.903382 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4569cce-33ba-4207-b8c8-798df2f15522 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:58.908040 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3bc7e-45da-47f4-985c-d85f2ac0cd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:58.908691 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3bc7e-45da-47f4-985c-d85f2ac0cd43 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:58.908976 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3bc7e-45da-47f4-985c-d85f2ac0cd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:58.909255 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3bc7e-45da-47f4-985c-d85f2ac0cd43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:58.910001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3bc7e-45da-47f4-985c-d85f2ac0cd43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:58.911609 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3bc7e-45da-47f4-985c-d85f2ac0cd43 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:58.922669 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1453/2858] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:58.926500 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2e948-29a8-44c9-b96e-57d3c46319b2 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:58.936418 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2e948-29a8-44c9-b96e-57d3c46319b2 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:58.941285 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd1fe6-1682-4f82-b88a-3f60f1ca1262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:58.941939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd1fe6-1682-4f82-b88a-3f60f1ca1262 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:58.942258 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd1fe6-1682-4f82-b88a-3f60f1ca1262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:58.942535 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd1fe6-1682-4f82-b88a-3f60f1ca1262 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:58.943255 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd1fe6-1682-4f82-b88a-3f60f1ca1262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:58.944827 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd1fe6-1682-4f82-b88a-3f60f1ca1262 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:58.947422 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:58.957190 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1406/2859] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:50:58.960803 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3bc7e-45da-47f4-985c-d85f2ac0cd43 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:58.971210 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3bc7e-45da-47f4-985c-d85f2ac0cd43 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:58.975805 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5d702-2615-494d-beef-7be2cc2c1ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:58.976666 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5d702-2615-494d-beef-7be2cc2c1ad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:58.976945 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5d702-2615-494d-beef-7be2cc2c1ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:58.977217 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5d702-2615-494d-beef-7be2cc2c1ad0 admin admin] PATH_INFO: `/v3/projects/18034700f491495583b5af434ad9ae8d/users/7d130b673f934c0e8ea122ea1706be01/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:58.991655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5d702-2615-494d-beef-7be2cc2c1ad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18034700f491495583b5af434ad9ae8d, user_id=7d130b673f934c0e8ea122ea1706be01, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:58.994428 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5d702-2615-494d-beef-7be2cc2c1ad0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:59.007726 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1454/2860] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:50:58 2022] PUT /identity/v3/projects/18034700f491495583b5af434ad9ae8d/users/7d130b673f934c0e8ea122ea1706be01/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:50:59.018206 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07728ce4-afb8-4ea6-a858-8667395092fc None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:59.018498 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07728ce4-afb8-4ea6-a858-8667395092fc None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:59.018778 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07728ce4-afb8-4ea6-a858-8667395092fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:59.045077 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-07728ce4-afb8-4ea6-a858-8667395092fc None None] MFA Rules not processed for user `7d130b673f934c0e8ea122ea1706be01`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:50:59.046358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-07728ce4-afb8-4ea6-a858-8667395092fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:59.108648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1407/2861] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:50:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:50:59.120326 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5d702-2615-494d-beef-7be2cc2c1ad0 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:50:59.122067 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-60c5d702-2615-494d-beef-7be2cc2c1ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:50:59.177542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5d702-2615-494d-beef-7be2cc2c1ad0 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:50:59.181641 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4accff6-a1cf-4779-a649-2123532cb93c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:50:59.182303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4accff6-a1cf-4779-a649-2123532cb93c service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:50:59.182583 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4accff6-a1cf-4779-a649-2123532cb93c service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:50:59.182851 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4accff6-a1cf-4779-a649-2123532cb93c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:50:59.187956 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:50:59.192673 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4accff6-a1cf-4779-a649-2123532cb93c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:50:59.194558 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4accff6-a1cf-4779-a649-2123532cb93c service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:50:59.204653 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1455/2862] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:50:59 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:51:09.105717 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=33 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:51:09.200802 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:51:15.081911 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28933baf-2cb7-4096-8bd4-2d01a464bc79 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:15.082430 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28933baf-2cb7-4096-8bd4-2d01a464bc79 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:15.082540 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28933baf-2cb7-4096-8bd4-2d01a464bc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:15.102669 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-28933baf-2cb7-4096-8bd4-2d01a464bc79 None None] MFA Rules not processed for user `dedf564ba5744a1fa70bb78222d26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:51:15.104509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-28933baf-2cb7-4096-8bd4-2d01a464bc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:51:15.284887 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1408/2863] 10.222.0.52 () {58 vars in 967 bytes} [Tue Jul 12 13:51:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:51:15.297001 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4accff6-a1cf-4779-a649-2123532cb93c service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:15.298888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b4accff6-a1cf-4779-a649-2123532cb93c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:51:15.368156 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4accff6-a1cf-4779-a649-2123532cb93c service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:15.373688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa460229-9c58-4f1b-b0c4-21b8ff51e6a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937bacab8c1c45c0876b8e61f6f3b859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:15.374354 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa460229-9c58-4f1b-b0c4-21b8ff51e6a5 service swift] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:15.374637 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa460229-9c58-4f1b-b0c4-21b8ff51e6a5 service swift] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:15.374911 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa460229-9c58-4f1b-b0c4-21b8ff51e6a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:15.383165 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa460229-9c58-4f1b-b0c4-21b8ff51e6a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:15.384823 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa460229-9c58-4f1b-b0c4-21b8ff51e6a5 service swift] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:15.392684 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1456/2864] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jul 12 13:51:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:51:22.238830 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28933baf-2cb7-4096-8bd4-2d01a464bc79 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:22.240284 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-28933baf-2cb7-4096-8bd4-2d01a464bc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:51:22.309119 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28933baf-2cb7-4096-8bd4-2d01a464bc79 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:22.313577 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973f422-6de6-4d3d-89af-d74d2cdd64c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:22.314471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973f422-6de6-4d3d-89af-d74d2cdd64c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:22.314751 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973f422-6de6-4d3d-89af-d74d2cdd64c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:22.315021 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973f422-6de6-4d3d-89af-d74d2cdd64c3 admin admin] PATH_INFO: `/v3/users/ab68f09ccf4d4ac9901d4ed792b551c0` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:22.316203 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8973f422-6de6-4d3d-89af-d74d2cdd64c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab68f09ccf4d4ac9901d4ed792b551c0)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:22.318140 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8973f422-6de6-4d3d-89af-d74d2cdd64c3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:22.364547 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8973f422-6de6-4d3d-89af-d74d2cdd64c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab68f09ccf4d4ac9901d4ed792b551c0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:22.376671 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8973f422-6de6-4d3d-89af-d74d2cdd64c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab68f09ccf4d4ac9901d4ed792b551c0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:22.383715 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8973f422-6de6-4d3d-89af-d74d2cdd64c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab68f09ccf4d4ac9901d4ed792b551c0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:22.384376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8973f422-6de6-4d3d-89af-d74d2cdd64c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab68f09ccf4d4ac9901d4ed792b551c0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:22.391597 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8973f422-6de6-4d3d-89af-d74d2cdd64c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab68f09ccf4d4ac9901d4ed792b551c0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:22.397466 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1409/2865] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:51:22 2022] DELETE /identity/v3/users/ab68f09ccf4d4ac9901d4ed792b551c0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:51:22.580713 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa460229-9c58-4f1b-b0c4-21b8ff51e6a5 service swift] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:22.592028 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa460229-9c58-4f1b-b0c4-21b8ff51e6a5 service swift] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:22.653429 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970e2aa-b3c5-4fda-b5d9-f15171a16580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:22.654314 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970e2aa-b3c5-4fda-b5d9-f15171a16580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:22.654602 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970e2aa-b3c5-4fda-b5d9-f15171a16580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:22.654879 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970e2aa-b3c5-4fda-b5d9-f15171a16580 admin admin] PATH_INFO: `/v3/projects/70562708174846e9bb7b1b128a629a98` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:22.656073 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970e2aa-b3c5-4fda-b5d9-f15171a16580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70562708174846e9bb7b1b128a629a98)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:22.658012 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970e2aa-b3c5-4fda-b5d9-f15171a16580 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:22.709151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d970e2aa-b3c5-4fda-b5d9-f15171a16580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70562708174846e9bb7b1b128a629a98', 'request_id': 'req-d970e2aa-b3c5-4fda-b5d9-f15171a16580'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:22.717735 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d970e2aa-b3c5-4fda-b5d9-f15171a16580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70562708174846e9bb7b1b128a629a98', 'request_id': 'req-d970e2aa-b3c5-4fda-b5d9-f15171a16580'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:22.724031 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d970e2aa-b3c5-4fda-b5d9-f15171a16580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70562708174846e9bb7b1b128a629a98', 'request_id': 'req-d970e2aa-b3c5-4fda-b5d9-f15171a16580'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:22.734888 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d970e2aa-b3c5-4fda-b5d9-f15171a16580 admin admin] The token cache is being invalidate because project 70562708174846e9bb7b1b128a629a98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:51:22.735177 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d970e2aa-b3c5-4fda-b5d9-f15171a16580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:22.735831 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d970e2aa-b3c5-4fda-b5d9-f15171a16580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:22.737378 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1457/2866] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:51:22 2022] DELETE /identity/v3/projects/70562708174846e9bb7b1b128a629a98 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:51:22.741355 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973f422-6de6-4d3d-89af-d74d2cdd64c3 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:22.742784 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8973f422-6de6-4d3d-89af-d74d2cdd64c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:51:22.756979 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973f422-6de6-4d3d-89af-d74d2cdd64c3 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:22.778568 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:51:22.817340 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769b5e9-ada3-45c4-b961-56e7b67769a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:22.818271 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769b5e9-ada3-45c4-b961-56e7b67769a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:22.818560 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769b5e9-ada3-45c4-b961-56e7b67769a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:22.818836 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769b5e9-ada3-45c4-b961-56e7b67769a3 admin admin] PATH_INFO: `/v3/users/90c83dec7921400ca1ec2e2cdbb38913` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:22.820101 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769b5e9-ada3-45c4-b961-56e7b67769a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90c83dec7921400ca1ec2e2cdbb38913)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:22.822599 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769b5e9-ada3-45c4-b961-56e7b67769a3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:22.873895 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b769b5e9-ada3-45c4-b961-56e7b67769a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90c83dec7921400ca1ec2e2cdbb38913'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:22.887109 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b769b5e9-ada3-45c4-b961-56e7b67769a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90c83dec7921400ca1ec2e2cdbb38913'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:22.893884 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b769b5e9-ada3-45c4-b961-56e7b67769a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90c83dec7921400ca1ec2e2cdbb38913'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:22.894628 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b769b5e9-ada3-45c4-b961-56e7b67769a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90c83dec7921400ca1ec2e2cdbb38913'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:22.903283 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b769b5e9-ada3-45c4-b961-56e7b67769a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90c83dec7921400ca1ec2e2cdbb38913'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:22.909927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1410/2867] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:51:22 2022] DELETE /identity/v3/users/90c83dec7921400ca1ec2e2cdbb38913 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:51:23.087306 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970e2aa-b3c5-4fda-b5d9-f15171a16580 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:23.089386 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d970e2aa-b3c5-4fda-b5d9-f15171a16580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:51:23.102869 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970e2aa-b3c5-4fda-b5d9-f15171a16580 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:23.123722 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:51:23.164984 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd6d6d-6ce5-4df8-b47e-384158496b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:23.165881 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd6d6d-6ce5-4df8-b47e-384158496b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:23.166206 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd6d6d-6ce5-4df8-b47e-384158496b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:23.166480 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd6d6d-6ce5-4df8-b47e-384158496b63 admin admin] PATH_INFO: `/v3/projects/dd37dadb214840cb87235a9b7e5acbff` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:23.167650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd6d6d-6ce5-4df8-b47e-384158496b63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd37dadb214840cb87235a9b7e5acbff)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:23.169545 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd6d6d-6ce5-4df8-b47e-384158496b63 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:23.216418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-11dd6d6d-6ce5-4df8-b47e-384158496b63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd37dadb214840cb87235a9b7e5acbff', 'request_id': 'req-11dd6d6d-6ce5-4df8-b47e-384158496b63'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:23.223323 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-11dd6d6d-6ce5-4df8-b47e-384158496b63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd37dadb214840cb87235a9b7e5acbff', 'request_id': 'req-11dd6d6d-6ce5-4df8-b47e-384158496b63'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:23.229504 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-11dd6d6d-6ce5-4df8-b47e-384158496b63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd37dadb214840cb87235a9b7e5acbff', 'request_id': 'req-11dd6d6d-6ce5-4df8-b47e-384158496b63'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:23.239622 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-11dd6d6d-6ce5-4df8-b47e-384158496b63 admin admin] The token cache is being invalidate because project dd37dadb214840cb87235a9b7e5acbff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:51:23.239909 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-11dd6d6d-6ce5-4df8-b47e-384158496b63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:23.240539 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-11dd6d6d-6ce5-4df8-b47e-384158496b63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:23.242106 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1458/2868] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:51:23 2022] DELETE /identity/v3/projects/dd37dadb214840cb87235a9b7e5acbff => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:51:32.906075 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:51:33.101735 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769b5e9-ada3-45c4-b961-56e7b67769a3 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:33.103860 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b769b5e9-ada3-45c4-b961-56e7b67769a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:51:33.116633 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769b5e9-ada3-45c4-b961-56e7b67769a3 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:33.171743 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de429edb-7639-45f1-8d6d-c56b01da0f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:33.172611 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de429edb-7639-45f1-8d6d-c56b01da0f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:33.172893 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de429edb-7639-45f1-8d6d-c56b01da0f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:33.173166 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de429edb-7639-45f1-8d6d-c56b01da0f5d admin admin] PATH_INFO: `/v3/users/7d130b673f934c0e8ea122ea1706be01` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:33.174376 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de429edb-7639-45f1-8d6d-c56b01da0f5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d130b673f934c0e8ea122ea1706be01)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:33.176266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de429edb-7639-45f1-8d6d-c56b01da0f5d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:33.220652 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-de429edb-7639-45f1-8d6d-c56b01da0f5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d130b673f934c0e8ea122ea1706be01'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:33.232699 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-de429edb-7639-45f1-8d6d-c56b01da0f5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d130b673f934c0e8ea122ea1706be01'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:33.236650 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:51:33.239500 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-de429edb-7639-45f1-8d6d-c56b01da0f5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d130b673f934c0e8ea122ea1706be01'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:33.240268 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-de429edb-7639-45f1-8d6d-c56b01da0f5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d130b673f934c0e8ea122ea1706be01'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:33.247389 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-de429edb-7639-45f1-8d6d-c56b01da0f5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d130b673f934c0e8ea122ea1706be01'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:33.253358 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1411/2869] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:51:33 2022] DELETE /identity/v3/users/7d130b673f934c0e8ea122ea1706be01 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:51:33.257807 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba50745-21ab-4421-92f3-cc071b6aeb86 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:33.258151 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba50745-21ab-4421-92f3-cc071b6aeb86 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:33.258438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba50745-21ab-4421-92f3-cc071b6aeb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:33.276914 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-6ba50745-21ab-4421-92f3-cc071b6aeb86 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:51:33.278200 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6ba50745-21ab-4421-92f3-cc071b6aeb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:51:33.336566 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1459/2870] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:51:33.347224 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de429edb-7639-45f1-8d6d-c56b01da0f5d admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:33.348607 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-de429edb-7639-45f1-8d6d-c56b01da0f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:51:33.403939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de429edb-7639-45f1-8d6d-c56b01da0f5d admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:33.407527 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af9656-62f7-4838-b3ae-485b22391659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:33.408160 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af9656-62f7-4838-b3ae-485b22391659 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:33.408439 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af9656-62f7-4838-b3ae-485b22391659 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:33.408710 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af9656-62f7-4838-b3ae-485b22391659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:33.417488 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af9656-62f7-4838-b3ae-485b22391659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:33.418961 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af9656-62f7-4838-b3ae-485b22391659 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:33.428121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1412/2871] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:51:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:51:33.938229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba50745-21ab-4421-92f3-cc071b6aeb86 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:33.939679 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6ba50745-21ab-4421-92f3-cc071b6aeb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:51:33.952254 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba50745-21ab-4421-92f3-cc071b6aeb86 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:33.957169 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616eec6-9683-4b8d-af30-a347e3b74356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:33.958287 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616eec6-9683-4b8d-af30-a347e3b74356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:33.958661 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616eec6-9683-4b8d-af30-a347e3b74356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:33.959251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616eec6-9683-4b8d-af30-a347e3b74356 admin admin] PATH_INFO: `/v3/projects/18034700f491495583b5af434ad9ae8d` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:33.960270 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616eec6-9683-4b8d-af30-a347e3b74356 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18034700f491495583b5af434ad9ae8d)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:33.963051 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616eec6-9683-4b8d-af30-a347e3b74356 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:34.010323 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3616eec6-9683-4b8d-af30-a347e3b74356 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18034700f491495583b5af434ad9ae8d', 'request_id': 'req-3616eec6-9683-4b8d-af30-a347e3b74356'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:34.017389 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3616eec6-9683-4b8d-af30-a347e3b74356 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18034700f491495583b5af434ad9ae8d', 'request_id': 'req-3616eec6-9683-4b8d-af30-a347e3b74356'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:34.023685 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3616eec6-9683-4b8d-af30-a347e3b74356 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18034700f491495583b5af434ad9ae8d', 'request_id': 'req-3616eec6-9683-4b8d-af30-a347e3b74356'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:34.034431 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3616eec6-9683-4b8d-af30-a347e3b74356 admin admin] The token cache is being invalidate because project 18034700f491495583b5af434ad9ae8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:51:34.034716 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3616eec6-9683-4b8d-af30-a347e3b74356 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:34.035343 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3616eec6-9683-4b8d-af30-a347e3b74356 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:34.036875 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1460/2872] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:51:33 2022] DELETE /identity/v3/projects/18034700f491495583b5af434ad9ae8d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:51:34.041646 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d5210-c17a-4dac-8df4-24ae229f9de2 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:34.042148 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d5210-c17a-4dac-8df4-24ae229f9de2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:34.042362 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d5210-c17a-4dac-8df4-24ae229f9de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:34.060141 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0d9d5210-c17a-4dac-8df4-24ae229f9de2 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:51:34.061332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0d9d5210-c17a-4dac-8df4-24ae229f9de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:51:34.077903 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:51:34.120683 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1413/2873] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:51:34.129591 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f2e99-7511-4b54-84bc-2ffcc417a8b0 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:34.129883 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f2e99-7511-4b54-84bc-2ffcc417a8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:34.130198 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f2e99-7511-4b54-84bc-2ffcc417a8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:34.148502 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-bb1f2e99-7511-4b54-84bc-2ffcc417a8b0 None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:51:34.150356 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-bb1f2e99-7511-4b54-84bc-2ffcc417a8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:51:34.158264 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1461/2874] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:51:34.162227 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d5210-c17a-4dac-8df4-24ae229f9de2 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:34.171812 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d5210-c17a-4dac-8df4-24ae229f9de2 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:34.176404 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc17c0f-62ef-4f88-9adc-987157112ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:34.177049 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc17c0f-62ef-4f88-9adc-987157112ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:34.177335 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc17c0f-62ef-4f88-9adc-987157112ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:34.177614 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc17c0f-62ef-4f88-9adc-987157112ed0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:34.178543 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc17c0f-62ef-4f88-9adc-987157112ed0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:51:34.178858 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc17c0f-62ef-4f88-9adc-987157112ed0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:34.180487 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc17c0f-62ef-4f88-9adc-987157112ed0 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:34.192096 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1414/2875] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jul 12 13:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:51:34.196150 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f2e99-7511-4b54-84bc-2ffcc417a8b0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:34.206182 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f2e99-7511-4b54-84bc-2ffcc417a8b0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:34.210892 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328511e1-f8c7-466c-a5c4-c2fa825dbf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:34.211536 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328511e1-f8c7-466c-a5c4-c2fa825dbf0c admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:34.211816 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328511e1-f8c7-466c-a5c4-c2fa825dbf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:34.212176 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328511e1-f8c7-466c-a5c4-c2fa825dbf0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:34.213424 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328511e1-f8c7-466c-a5c4-c2fa825dbf0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:34.215369 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328511e1-f8c7-466c-a5c4-c2fa825dbf0c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:34.239956 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1462/2876] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jul 12 13:51:34 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:51:34.244048 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc17c0f-62ef-4f88-9adc-987157112ed0 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:34.254142 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc17c0f-62ef-4f88-9adc-987157112ed0 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:34.310178 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307218b-5af9-4708-b0f6-33075bd6164a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:34.310849 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307218b-5af9-4708-b0f6-33075bd6164a admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:34.311136 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307218b-5af9-4708-b0f6-33075bd6164a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:34.311414 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307218b-5af9-4708-b0f6-33075bd6164a admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:34.312110 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9307218b-5af9-4708-b0f6-33075bd6164a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:34.314036 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9307218b-5af9-4708-b0f6-33075bd6164a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:34.333579 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1415/2877] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jul 12 13:51:34 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:51:34.338542 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328511e1-f8c7-466c-a5c4-c2fa825dbf0c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:34.348776 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328511e1-f8c7-466c-a5c4-c2fa825dbf0c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:34.353472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbb067-925e-458c-a930-37ad1b68bbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:34.354166 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbb067-925e-458c-a930-37ad1b68bbfb admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:34.354449 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbb067-925e-458c-a930-37ad1b68bbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:34.354722 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbb067-925e-458c-a930-37ad1b68bbfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:34.355438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbb067-925e-458c-a930-37ad1b68bbfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:34.357047 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbb067-925e-458c-a930-37ad1b68bbfb admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:34.368108 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1463/2878] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:51:34.372470 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307218b-5af9-4708-b0f6-33075bd6164a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:34.382702 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307218b-5af9-4708-b0f6-33075bd6164a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:34.388006 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d084375-46de-47f4-8f90-ddae41b9597d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:34.388649 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d084375-46de-47f4-8f90-ddae41b9597d admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:34.388927 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d084375-46de-47f4-8f90-ddae41b9597d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:34.389197 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d084375-46de-47f4-8f90-ddae41b9597d admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:34.389911 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d084375-46de-47f4-8f90-ddae41b9597d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:34.391514 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d084375-46de-47f4-8f90-ddae41b9597d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:34.401062 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1416/2879] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jul 12 13:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:51:34.405264 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbb067-925e-458c-a930-37ad1b68bbfb admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:34.415456 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbb067-925e-458c-a930-37ad1b68bbfb admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:34.420634 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9729473-8b46-41eb-b77d-bc7b74e29307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:34.421510 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9729473-8b46-41eb-b77d-bc7b74e29307 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:34.421789 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9729473-8b46-41eb-b77d-bc7b74e29307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:34.422097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9729473-8b46-41eb-b77d-bc7b74e29307 admin admin] PATH_INFO: `/v3/projects/eb8f2d409d55434eb8d1c05f0b9f5b84/users/99f378f099a64fba8b556ed4a93ebc68/roles/ed842da869724411a5b355cd12f1c5c8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:34.437045 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9729473-8b46-41eb-b77d-bc7b74e29307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb8f2d409d55434eb8d1c05f0b9f5b84, user_id=99f378f099a64fba8b556ed4a93ebc68, role_id=ed842da869724411a5b355cd12f1c5c8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:34.439709 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9729473-8b46-41eb-b77d-bc7b74e29307 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:34.452766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1464/2880] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jul 12 13:51:34 2022] PUT /identity/v3/projects/eb8f2d409d55434eb8d1c05f0b9f5b84/users/99f378f099a64fba8b556ed4a93ebc68/roles/ed842da869724411a5b355cd12f1c5c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:51:34.463032 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d892a90-9a1d-4667-9933-2325e4d04916 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:34.463316 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d892a90-9a1d-4667-9933-2325e4d04916 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:34.463588 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d892a90-9a1d-4667-9933-2325e4d04916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:34.490117 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8d892a90-9a1d-4667-9933-2325e4d04916 None None] MFA Rules not processed for user `99f378f099a64fba8b556ed4a93ebc68`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:51:34.491370 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8d892a90-9a1d-4667-9933-2325e4d04916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:51:34.552810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1417/2881] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:51:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:51:34.564438 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9729473-8b46-41eb-b77d-bc7b74e29307 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:34.566208 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f9729473-8b46-41eb-b77d-bc7b74e29307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:51:34.588866 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:51:34.624844 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9729473-8b46-41eb-b77d-bc7b74e29307 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:34.628775 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7680064-7931-416b-a2ab-a344dd470d60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35336384bbca475082be349df406ae3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:34.629418 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7680064-7931-416b-a2ab-a344dd470d60 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:34.629698 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7680064-7931-416b-a2ab-a344dd470d60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:34.629982 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7680064-7931-416b-a2ab-a344dd470d60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:34.638262 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7680064-7931-416b-a2ab-a344dd470d60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:34.639702 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7680064-7931-416b-a2ab-a344dd470d60 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:34.648461 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1465/2882] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:51:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:51:34.674569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d892a90-9a1d-4667-9933-2325e4d04916 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:34.693264 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:51:34.737145 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d892a90-9a1d-4667-9933-2325e4d04916 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:34.741903 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3668b1-1948-4735-9ed3-8ab5609c1429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:34.742797 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3668b1-1948-4735-9ed3-8ab5609c1429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:34.743073 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3668b1-1948-4735-9ed3-8ab5609c1429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:34.743345 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3668b1-1948-4735-9ed3-8ab5609c1429 admin admin] PATH_INFO: `/v3/users/99f378f099a64fba8b556ed4a93ebc68` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:34.744569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3668b1-1948-4735-9ed3-8ab5609c1429 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99f378f099a64fba8b556ed4a93ebc68)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:34.746506 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3668b1-1948-4735-9ed3-8ab5609c1429 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:34.791346 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2b3668b1-1948-4735-9ed3-8ab5609c1429 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99f378f099a64fba8b556ed4a93ebc68'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:34.802892 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2b3668b1-1948-4735-9ed3-8ab5609c1429 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99f378f099a64fba8b556ed4a93ebc68'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:34.809536 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2b3668b1-1948-4735-9ed3-8ab5609c1429 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99f378f099a64fba8b556ed4a93ebc68'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:34.810220 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2b3668b1-1948-4735-9ed3-8ab5609c1429 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99f378f099a64fba8b556ed4a93ebc68'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:34.817336 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2b3668b1-1948-4735-9ed3-8ab5609c1429 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99f378f099a64fba8b556ed4a93ebc68'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:34.823692 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1418/2883] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:51:34 2022] DELETE /identity/v3/users/99f378f099a64fba8b556ed4a93ebc68 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:51:34.828215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a34f8-bd0d-42ba-af3c-a7c66ec35b2b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:34.828511 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a34f8-bd0d-42ba-af3c-a7c66ec35b2b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:34.828820 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a34f8-bd0d-42ba-af3c-a7c66ec35b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:34.846861 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4f3a34f8-bd0d-42ba-af3c-a7c66ec35b2b None None] MFA Rules not processed for user `0ed92bd32dc24cfabfcf0a70a4518dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:51:34.848109 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4f3a34f8-bd0d-42ba-af3c-a7c66ec35b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:51:34.906473 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1466/2884] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jul 12 13:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:51:34.916227 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3668b1-1948-4735-9ed3-8ab5609c1429 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:34.917612 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2b3668b1-1948-4735-9ed3-8ab5609c1429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:51:34.974768 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3668b1-1948-4735-9ed3-8ab5609c1429 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:34.978594 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206cb15-7d76-4fa5-9c48-a05e839fc528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:34.979180 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206cb15-7d76-4fa5-9c48-a05e839fc528 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:34.979472 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206cb15-7d76-4fa5-9c48-a05e839fc528 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:34.979752 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206cb15-7d76-4fa5-9c48-a05e839fc528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:34.989611 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206cb15-7d76-4fa5-9c48-a05e839fc528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:34.991609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206cb15-7d76-4fa5-9c48-a05e839fc528 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:35.001229 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1419/2885] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:51:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:51:35.264224 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a34f8-bd0d-42ba-af3c-a7c66ec35b2b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:51:35.265831 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4f3a34f8-bd0d-42ba-af3c-a7c66ec35b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:51:35.280472 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a34f8-bd0d-42ba-af3c-a7c66ec35b2b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:51:35.286158 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af54b7-eae1-49b6-90fd-31ad09da6249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:51:35.287023 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af54b7-eae1-49b6-90fd-31ad09da6249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:51:35.287303 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af54b7-eae1-49b6-90fd-31ad09da6249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:51:35.287559 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af54b7-eae1-49b6-90fd-31ad09da6249 admin admin] PATH_INFO: `/v3/projects/eb8f2d409d55434eb8d1c05f0b9f5b84` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:51:35.288749 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af54b7-eae1-49b6-90fd-31ad09da6249 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb8f2d409d55434eb8d1c05f0b9f5b84)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:51:35.290789 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af54b7-eae1-49b6-90fd-31ad09da6249 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:51:35.336446 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-06af54b7-eae1-49b6-90fd-31ad09da6249 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb8f2d409d55434eb8d1c05f0b9f5b84', 'request_id': 'req-06af54b7-eae1-49b6-90fd-31ad09da6249'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:35.343376 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-06af54b7-eae1-49b6-90fd-31ad09da6249 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb8f2d409d55434eb8d1c05f0b9f5b84', 'request_id': 'req-06af54b7-eae1-49b6-90fd-31ad09da6249'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:35.349543 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-06af54b7-eae1-49b6-90fd-31ad09da6249 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb8f2d409d55434eb8d1c05f0b9f5b84', 'request_id': 'req-06af54b7-eae1-49b6-90fd-31ad09da6249'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:35.359945 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-06af54b7-eae1-49b6-90fd-31ad09da6249 admin admin] The token cache is being invalidate because project eb8f2d409d55434eb8d1c05f0b9f5b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:51:35.360233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-06af54b7-eae1-49b6-90fd-31ad09da6249 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:35.360867 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-06af54b7-eae1-49b6-90fd-31ad09da6249 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:51:35.362436 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1467/2886] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:51:35 2022] DELETE /identity/v3/projects/eb8f2d409d55434eb8d1c05f0b9f5b84 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:51:44.996970 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:51:45.356897 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:52:02.411939 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206cb15-7d76-4fa5-9c48-a05e839fc528 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:52:02.421116 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206cb15-7d76-4fa5-9c48-a05e839fc528 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:52:02.470757 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e4dc6-bce4-4d75-b90e-87acf98bf00e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:52:02.471404 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e4dc6-bce4-4d75-b90e-87acf98bf00e service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:52:02.471729 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e4dc6-bce4-4d75-b90e-87acf98bf00e service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:52:02.472012 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e4dc6-bce4-4d75-b90e-87acf98bf00e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:52:02.473821 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1d4e4dc6-bce4-4d75-b90e-87acf98bf00e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:52:02.545609 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e4dc6-bce4-4d75-b90e-87acf98bf00e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:52:02.547103 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e4dc6-bce4-4d75-b90e-87acf98bf00e service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:52:02.558989 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1420/2887] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:52:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:52:03.735518 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af54b7-eae1-49b6-90fd-31ad09da6249 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:52:03.736931 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-06af54b7-eae1-49b6-90fd-31ad09da6249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:52:03.749516 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af54b7-eae1-49b6-90fd-31ad09da6249 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:52:03.753337 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33430587-c063-4fb9-a20b-823682e391fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:52:03.753902 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33430587-c063-4fb9-a20b-823682e391fd service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:52:03.754223 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33430587-c063-4fb9-a20b-823682e391fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:52:03.754496 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33430587-c063-4fb9-a20b-823682e391fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:52:03.756202 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-33430587-c063-4fb9-a20b-823682e391fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:52:03.770355 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33430587-c063-4fb9-a20b-823682e391fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:52:03.771842 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33430587-c063-4fb9-a20b-823682e391fd service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:52:03.783056 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1468/2888] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:52:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:52:12.553390 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=54 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:52:13.778076 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=6 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:53:03.482831 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e4dc6-bce4-4d75-b90e-87acf98bf00e service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:53:03.492275 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e4dc6-bce4-4d75-b90e-87acf98bf00e service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:53:03.495868 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6756a-119f-4435-9bdd-7ccecf5d1c5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:53:03.496531 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6756a-119f-4435-9bdd-7ccecf5d1c5e service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:53:03.496813 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6756a-119f-4435-9bdd-7ccecf5d1c5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:53:03.497088 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6756a-119f-4435-9bdd-7ccecf5d1c5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:53:03.498873 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-00e6756a-119f-4435-9bdd-7ccecf5d1c5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:53:03.508723 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6756a-119f-4435-9bdd-7ccecf5d1c5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:53:03.510165 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6756a-119f-4435-9bdd-7ccecf5d1c5e service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:53:03.518157 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1421/2889] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:53:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:53:13.514502 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:53:41.548126 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33430587-c063-4fb9-a20b-823682e391fd service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:53:41.550702 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-33430587-c063-4fb9-a20b-823682e391fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:53:41.565733 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33430587-c063-4fb9-a20b-823682e391fd service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:53:41.574715 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae877fd-f5b0-4099-8c95-5714d1d6a88b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:53:41.575379 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae877fd-f5b0-4099-8c95-5714d1d6a88b service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:53:41.575660 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae877fd-f5b0-4099-8c95-5714d1d6a88b service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:53:41.575934 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae877fd-f5b0-4099-8c95-5714d1d6a88b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:53:41.577766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7ae877fd-f5b0-4099-8c95-5714d1d6a88b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:53:41.591244 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae877fd-f5b0-4099-8c95-5714d1d6a88b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:53:41.593003 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae877fd-f5b0-4099-8c95-5714d1d6a88b service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:53:41.605288 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1469/2890] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:53:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:53:43.758958 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6756a-119f-4435-9bdd-7ccecf5d1c5e service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:53:43.773651 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6756a-119f-4435-9bdd-7ccecf5d1c5e service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:53:43.779851 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072578c-bc02-4d4e-84cf-e1b4439e8cf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3915cde0d04bcfb9b5ce8c89af1e5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:53:43.780523 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072578c-bc02-4d4e-84cf-e1b4439e8cf9 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:53:43.780808 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072578c-bc02-4d4e-84cf-e1b4439e8cf9 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:53:43.781085 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072578c-bc02-4d4e-84cf-e1b4439e8cf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:53:43.783648 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8072578c-bc02-4d4e-84cf-e1b4439e8cf9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:53:43.796604 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072578c-bc02-4d4e-84cf-e1b4439e8cf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:53:43.798227 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072578c-bc02-4d4e-84cf-e1b4439e8cf9 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:53:43.811171 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1422/2891] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:53:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:53:45.396785 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae877fd-f5b0-4099-8c95-5714d1d6a88b service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:53:45.407312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae877fd-f5b0-4099-8c95-5714d1d6a88b service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:53:45.410886 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7027894-60f6-4c7b-9928-44ed4b69095b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e833722fe4e493e9ddc4b275632536f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:53:45.411528 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7027894-60f6-4c7b-9928-44ed4b69095b service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:53:45.411814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7027894-60f6-4c7b-9928-44ed4b69095b service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:53:45.412091 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7027894-60f6-4c7b-9928-44ed4b69095b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:53:45.413864 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e7027894-60f6-4c7b-9928-44ed4b69095b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:53:45.477815 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7027894-60f6-4c7b-9928-44ed4b69095b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:53:45.479229 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7027894-60f6-4c7b-9928-44ed4b69095b service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:53:45.489655 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1470/2892] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:53:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:53:48.677810 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072578c-bc02-4d4e-84cf-e1b4439e8cf9 service nova] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:53:48.679332 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8072578c-bc02-4d4e-84cf-e1b4439e8cf9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:53:48.781743 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072578c-bc02-4d4e-84cf-e1b4439e8cf9 service nova] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:53:48.787239 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc75ea2-80c4-459a-a659-db722ec03dea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1308deea1ea4629a1ac22d4b903ad4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:53:48.787902 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc75ea2-80c4-459a-a659-db722ec03dea service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:53:48.788192 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc75ea2-80c4-459a-a659-db722ec03dea service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:53:48.788470 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc75ea2-80c4-459a-a659-db722ec03dea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:53:48.790562 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ecc75ea2-80c4-459a-a659-db722ec03dea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:53:48.805432 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc75ea2-80c4-459a-a659-db722ec03dea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:53:48.807070 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc75ea2-80c4-459a-a659-db722ec03dea service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:53:48.818776 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1423/2893] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:53:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:53:51.696312 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7027894-60f6-4c7b-9928-44ed4b69095b service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:53:51.697993 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e7027894-60f6-4c7b-9928-44ed4b69095b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:53:51.715646 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7027894-60f6-4c7b-9928-44ed4b69095b service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:53:51.721209 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31edff-890e-4778-9fee-820d5ffb67f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:53:51.722153 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31edff-890e-4778-9fee-820d5ffb67f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:53:51.722484 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31edff-890e-4778-9fee-820d5ffb67f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:53:51.722725 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31edff-890e-4778-9fee-820d5ffb67f4 admin admin] PATH_INFO: `/v3/users/3dd90f756d724016b60c80e9c78cce41` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:53:51.723916 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31edff-890e-4778-9fee-820d5ffb67f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dd90f756d724016b60c80e9c78cce41)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:53:51.725838 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31edff-890e-4778-9fee-820d5ffb67f4 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:53:51.773679 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6d31edff-890e-4778-9fee-820d5ffb67f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dd90f756d724016b60c80e9c78cce41'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:51.774353 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6d31edff-890e-4778-9fee-820d5ffb67f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dd90f756d724016b60c80e9c78cce41'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:51.774959 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6d31edff-890e-4778-9fee-820d5ffb67f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dd90f756d724016b60c80e9c78cce41'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:51.787688 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6d31edff-890e-4778-9fee-820d5ffb67f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dd90f756d724016b60c80e9c78cce41'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:51.794456 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6d31edff-890e-4778-9fee-820d5ffb67f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dd90f756d724016b60c80e9c78cce41'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:51.807003 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1471/2894] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:53:51 2022] DELETE /identity/v3/users/3dd90f756d724016b60c80e9c78cce41 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:53:51.980126 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc75ea2-80c4-459a-a659-db722ec03dea service placement] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:53:51.992716 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc75ea2-80c4-459a-a659-db722ec03dea service placement] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:53:52.065069 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee8292-6dbb-4875-bc6e-4b57cb223a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:53:52.065935 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee8292-6dbb-4875-bc6e-4b57cb223a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:53:52.066251 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee8292-6dbb-4875-bc6e-4b57cb223a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:53:52.066531 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee8292-6dbb-4875-bc6e-4b57cb223a09 admin admin] PATH_INFO: `/v3/projects/dc981a893df54f079e63399ace23ef1c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:53:52.067711 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee8292-6dbb-4875-bc6e-4b57cb223a09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc981a893df54f079e63399ace23ef1c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:53:52.069602 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee8292-6dbb-4875-bc6e-4b57cb223a09 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:53:52.120435 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-13ee8292-6dbb-4875-bc6e-4b57cb223a09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc981a893df54f079e63399ace23ef1c', 'request_id': 'req-13ee8292-6dbb-4875-bc6e-4b57cb223a09'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:52.127050 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-13ee8292-6dbb-4875-bc6e-4b57cb223a09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc981a893df54f079e63399ace23ef1c', 'request_id': 'req-13ee8292-6dbb-4875-bc6e-4b57cb223a09'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:52.134509 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-13ee8292-6dbb-4875-bc6e-4b57cb223a09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc981a893df54f079e63399ace23ef1c', 'request_id': 'req-13ee8292-6dbb-4875-bc6e-4b57cb223a09'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:52.146535 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-13ee8292-6dbb-4875-bc6e-4b57cb223a09 admin admin] The token cache is being invalidate because project dc981a893df54f079e63399ace23ef1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:53:52.146847 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-13ee8292-6dbb-4875-bc6e-4b57cb223a09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:52.147508 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-13ee8292-6dbb-4875-bc6e-4b57cb223a09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:52.149098 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1424/2895] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:53:51 2022] DELETE /identity/v3/projects/dc981a893df54f079e63399ace23ef1c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:53:52.153328 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31edff-890e-4778-9fee-820d5ffb67f4 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:53:52.154707 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6d31edff-890e-4778-9fee-820d5ffb67f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:53:52.167814 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31edff-890e-4778-9fee-820d5ffb67f4 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:53:52.215876 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:53:52.225766 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9cb50-d60b-43f7-9930-382196d90efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:53:52.226699 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9cb50-d60b-43f7-9930-382196d90efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:53:52.227215 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9cb50-d60b-43f7-9930-382196d90efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:53:52.227493 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9cb50-d60b-43f7-9930-382196d90efc admin admin] PATH_INFO: `/v3/users/9fb6b21121b947658272e0f195a79f01` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:53:52.228686 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9cb50-d60b-43f7-9930-382196d90efc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fb6b21121b947658272e0f195a79f01)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:53:52.230714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9cb50-d60b-43f7-9930-382196d90efc admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:53:52.275825 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d7b9cb50-d60b-43f7-9930-382196d90efc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fb6b21121b947658272e0f195a79f01'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:52.276488 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d7b9cb50-d60b-43f7-9930-382196d90efc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fb6b21121b947658272e0f195a79f01'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:52.277098 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d7b9cb50-d60b-43f7-9930-382196d90efc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fb6b21121b947658272e0f195a79f01'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:52.289002 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d7b9cb50-d60b-43f7-9930-382196d90efc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fb6b21121b947658272e0f195a79f01'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:52.295761 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d7b9cb50-d60b-43f7-9930-382196d90efc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fb6b21121b947658272e0f195a79f01'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:52.309091 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1472/2896] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jul 12 13:53:52 2022] DELETE /identity/v3/users/9fb6b21121b947658272e0f195a79f01 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:53:52.482955 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee8292-6dbb-4875-bc6e-4b57cb223a09 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:53:52.484569 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-13ee8292-6dbb-4875-bc6e-4b57cb223a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:53:52.497784 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee8292-6dbb-4875-bc6e-4b57cb223a09 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:53:52.504471 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:53:52.561110 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53046374-c526-4ffb-85a5-7bd5ff5b33ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed92bd32dc24cfabfcf0a70a4518dcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e45bcca33094251b8a3ed60293c2eb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:53:52.562001 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53046374-c526-4ffb-85a5-7bd5ff5b33ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:53:52.562988 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53046374-c526-4ffb-85a5-7bd5ff5b33ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:53:52.563266 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53046374-c526-4ffb-85a5-7bd5ff5b33ed admin admin] PATH_INFO: `/v3/projects/5b8c618fc5664694ad0bb1f5fb414ec1` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:53:52.564460 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53046374-c526-4ffb-85a5-7bd5ff5b33ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b8c618fc5664694ad0bb1f5fb414ec1)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:53:52.566398 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53046374-c526-4ffb-85a5-7bd5ff5b33ed admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:53:52.610773 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-53046374-c526-4ffb-85a5-7bd5ff5b33ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b8c618fc5664694ad0bb1f5fb414ec1', 'request_id': 'req-53046374-c526-4ffb-85a5-7bd5ff5b33ed'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:52.616613 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-53046374-c526-4ffb-85a5-7bd5ff5b33ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b8c618fc5664694ad0bb1f5fb414ec1', 'request_id': 'req-53046374-c526-4ffb-85a5-7bd5ff5b33ed'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:52.623908 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-53046374-c526-4ffb-85a5-7bd5ff5b33ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b8c618fc5664694ad0bb1f5fb414ec1', 'request_id': 'req-53046374-c526-4ffb-85a5-7bd5ff5b33ed'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:52.635683 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-53046374-c526-4ffb-85a5-7bd5ff5b33ed admin admin] The token cache is being invalidate because project 5b8c618fc5664694ad0bb1f5fb414ec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:53:52.635966 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-53046374-c526-4ffb-85a5-7bd5ff5b33ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:52.636583 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-53046374-c526-4ffb-85a5-7bd5ff5b33ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:53:52.638121 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1425/2897] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jul 12 13:53:52 2022] DELETE /identity/v3/projects/5b8c618fc5664694ad0bb1f5fb414ec1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:54:02.305259 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:54:02.632753 dv-cinder-797970-4-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:55:08.713214 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9cb50-d60b-43f7-9930-382196d90efc admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:55:08.714680 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d7b9cb50-d60b-43f7-9930-382196d90efc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:55:08.786409 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9cb50-d60b-43f7-9930-382196d90efc admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:55:08.845251 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0d0f7-9920-475a-baaa-069f377e64df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1308deea1ea4629a1ac22d4b903ad4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36aa0326fd0545dface07a9ddc308ccd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:55:08.845917 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0d0f7-9920-475a-baaa-069f377e64df service placement] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:55:08.846233 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0d0f7-9920-475a-baaa-069f377e64df service placement] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:55:08.847041 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0d0f7-9920-475a-baaa-069f377e64df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:55:08.848841 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b0d0d0f7-9920-475a-baaa-069f377e64df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:55:08.860714 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0d0f7-9920-475a-baaa-069f377e64df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:55:08.862946 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0d0f7-9920-475a-baaa-069f377e64df service placement] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:55:08.873097 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1473/2898] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jul 12 13:55:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:55:18.868715 dv-cinder-797970-4-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=58 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}